-- Logs begin at Sun 2022-09-11 00:10:31 UTC, end at Sun 2022-09-11 01:11:42 UTC. -- Sep 11 00:43:54.497649 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 11 00:43:54.505009 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 00:43:54 2022] *** Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: nodename: dv-cinder-master-3-iscsi Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: machine: x86_64 Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: clock source: unix Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: pcre jit disabled Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: detected number of CPU cores: 4 Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: current working directory: / Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: detected binary path: /usr/bin/uwsgi-core Sep 11 00:43:54.508094 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: your processes number limit is 47680 Sep 11 00:43:54.508805 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: your memory page size is 4096 bytes Sep 11 00:43:54.508805 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: detected max file descriptor number: 2048 Sep 11 00:43:54.508805 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: lock engine: pthread robust mutexes Sep 11 00:43:54.508805 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: thunder lock: enabled Sep 11 00:43:54.508805 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 11 00:43:54.508805 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 00:43:54.548760 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: Python main interpreter initialized at 0x55cd431bc9c0 Sep 11 00:43:54.548760 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: python threads support enabled Sep 11 00:43:54.548760 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: your server socket listen backlog is limited to 100 connections Sep 11 00:43:54.548760 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: your mercy for graceful operations on workers is 90 seconds Sep 11 00:43:54.549151 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 00:43:54.549234 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: *** Operational MODE: preforking *** Sep 11 00:43:54.549439 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: *** uWSGI is running in multiple interpreter mode *** Sep 11 00:43:54.549439 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: spawned uWSGI master process (pid: 111936) Sep 11 00:43:54.549639 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 11 00:43:54.554812 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: spawned uWSGI worker 1 (pid: 111937, cores: 1) Sep 11 00:43:54.554812 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: spawned uWSGI worker 2 (pid: 111938, cores: 1) Sep 11 00:43:54.554812 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 00:43:55.209281 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 11 00:43:55.225417 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 11 00:43:56.293844 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:43:56.293844 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: from cryptography.utils import int_from_bytes Sep 11 00:43:56.294452 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:43:56.294452 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: from cryptography.utils import int_from_bytes Sep 11 00:43:56.297666 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:43:56.297666 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: from cryptography.utils import int_from_bytes Sep 11 00:43:56.298094 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:43:56.298094 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: from cryptography.utils import int_from_bytes Sep 11 00:43:58.803754 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 00:43:58.804336 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 00:43:58.804526 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 00:43:58.804891 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 00:43:58.805147 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 00:43:58.805535 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.805865 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.805971 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 00:43:58.806330 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 00:43:58.806422 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.806512 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.806661 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 00:43:58.806830 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.807403 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 00:43:58.807510 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.807510 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.807656 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 00:43:58.807832 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.807934 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.808163 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.808249 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.808428 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.808537 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.808746 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.808858 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.809075 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.809191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.809399 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.809505 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.809708 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.809826 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810023 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810145 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810343 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810473 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810650 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810745 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.810999 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.811087 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.811308 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.811399 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.811682 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.811810 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.811990 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.812238 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.812366 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.812550 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.812661 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.812865 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.812961 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.813197 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.813297 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.813513 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.813611 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.813826 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.814011 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.814131 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.814320 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.814465 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.814661 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.814915 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.815103 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.815299 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.815456 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.815669 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.815861 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816045 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816201 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816419 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816611 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816798 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816982 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.816982 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.817297 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.817480 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.817697 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.817697 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.817998 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.818124 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.818368 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.818511 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.818659 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.818799 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.818946 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.819072 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.819195 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.819348 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.819481 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.819611 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.819866 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820007 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820132 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820254 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820383 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820515 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820677 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820802 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.820930 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821053 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821174 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821378 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821503 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821624 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821745 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.821878 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.822000 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.822142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.822264 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.822385 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.822513 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.823248 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.823451 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.823577 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.823704 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.823827 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.823957 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824082 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824215 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824368 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824499 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824631 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824801 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.824945 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.825116 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.825252 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.825425 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.825558 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.825736 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.825877 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826002 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826126 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826311 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826446 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826631 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826760 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.826966 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.827091 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.827272 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.827414 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.827580 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.827707 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.827886 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828013 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828192 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828325 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828497 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828624 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828802 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.828935 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.829108 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.829234 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.829426 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.829558 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.829740 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830009 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830137 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830275 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830400 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830530 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830653 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.830776 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831007 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831150 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831307 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831432 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831615 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831740 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.831924 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.832061 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.832244 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.832378 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.832551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.832676 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.832869 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833006 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833135 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833257 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833409 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833536 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833719 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.833853 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.834040 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.834167 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.834348 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.834475 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.834656 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.834847 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835000 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835128 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835251 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835371 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835538 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835665 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835843 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.835968 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.836161 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.836288 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.836467 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.836604 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.836773 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.836900 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837077 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837201 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837377 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837509 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837686 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837811 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.837992 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.838125 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.838857 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.838983 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.839188 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.839475 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.839627 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.839798 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.839949 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.840147 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.840283 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.840408 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.840530 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.840695 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.840823 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841004 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841139 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841310 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841436 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841616 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841748 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.841938 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.842064 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.842245 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.842375 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.842550 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.842675 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.842880 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843020 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843190 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843315 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843501 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843628 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843804 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.843928 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.844109 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.844233 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.844418 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.844558 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.844721 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.844853 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845033 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845160 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845337 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845463 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845644 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845783 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.845947 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.846084 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.846250 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.846374 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.846547 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.846684 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.846873 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847015 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847182 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847309 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847484 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847618 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847787 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.847927 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.848089 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.848215 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.848389 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.848515 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.848692 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.848900 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849015 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849143 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849273 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849397 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849573 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849700 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.849872 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.849998 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.850192 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.850324 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.850512 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.850615 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.850842 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.850934 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.851154 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.851241 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.851460 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.851550 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.851779 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:43:58.851874 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.852113 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.852199 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.852446 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.852535 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.852778 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.852864 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.853094 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.853197 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.853410 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.853497 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.853731 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.853835 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.854096 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.854205 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.854429 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.854710 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.854957 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.855090 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.855287 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.855408 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.855612 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.855726 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.855929 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856041 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856250 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856362 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856584 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856687 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856909 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.856996 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.857227 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.857323 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.857574 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.857661 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.857889 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.857975 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.858207 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.858302 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.858519 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.858613 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.858856 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.858959 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.859187 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.859288 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.859504 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.859599 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.859814 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.859900 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.860164 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.860252 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.860471 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.860573 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.860785 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.860872 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.861108 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.861203 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.861423 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.861510 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.861733 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.861864 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.862044 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.862138 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.862355 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.862441 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.862667 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.862761 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863019 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863105 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863331 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863426 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863646 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863745 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.863959 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864053 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864282 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864368 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864595 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864694 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864908 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.864994 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.865218 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.865303 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.865529 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.865624 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.865839 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.865926 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.866155 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.866242 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.866463 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.866558 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.866771 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.866899 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.867102 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.867189 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.867412 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.867500 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.867718 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.867810 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868026 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868111 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868339 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868427 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868649 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868743 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.868954 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.869040 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.869261 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.869371 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.869611 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.869698 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.870127 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.870261 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.870477 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.870634 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.870815 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.870986 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.871142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.871318 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.871456 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.871629 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.871768 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.871964 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.872096 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.872263 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.872413 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.872567 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.872721 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.872880 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873048 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873190 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873358 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873494 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873667 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873805 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.873986 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.874115 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.874301 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.874421 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.874614 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.874743 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.874955 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.875085 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.875268 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.875394 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.875594 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.875718 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.875911 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876023 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876224 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876330 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876561 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876648 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876894 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.876981 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.877201 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.877287 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.877531 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.877644 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.877839 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.877945 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.878159 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.878247 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.878464 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.878558 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.878770 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.878876 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.879100 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.879184 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.879411 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.879515 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.879722 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.879808 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880033 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880119 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880340 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880464 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880661 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880754 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.880950 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881037 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881254 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881350 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881556 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881643 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881860 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.881991 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.882161 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.882288 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.882468 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.882594 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.882768 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.882925 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883089 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883225 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883391 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883586 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883711 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883842 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.883979 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.884103 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.884326 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.884468 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.884643 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.884769 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.884952 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.885638 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.885894 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.886021 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.886146 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.886453 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.886591 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.886763 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.886910 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.887100 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.887230 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.887416 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.887551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.887740 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.887868 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.888054 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.888182 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.888376 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.888510 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.888691 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.888817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889013 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889138 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889325 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889459 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889636 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889768 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.889948 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.890084 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.890260 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.890387 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.890581 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.890720 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.890932 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891074 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891230 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891359 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891541 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891674 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891858 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.891992 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.892164 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.892289 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.892474 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.892606 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.892778 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.892915 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.893089 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.893214 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.893400 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.893562 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.893719 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.893863 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894032 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894159 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894349 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894475 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894657 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894812 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.894996 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.895122 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.895305 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.895432 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.895616 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.895750 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.895921 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896045 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896227 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896358 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896542 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896678 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.896978 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.897169 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.897295 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.897478 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.897605 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.897798 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.897930 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.898105 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.898243 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.898415 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.898548 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.898715 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.898863 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899048 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899190 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899356 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899483 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899668 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899794 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.899991 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.900126 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.900303 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.900437 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.900616 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.900845 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.901109 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.901256 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.901382 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.901574 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.901707 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.901882 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902091 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902218 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902348 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902473 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902604 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902830 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.902975 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.903136 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.903264 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.903462 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.903597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.903791 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.903933 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.904101 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.904228 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.904409 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.904529 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.904718 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.904853 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905023 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905162 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905327 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905452 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905633 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905761 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.905941 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.906077 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.906250 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.906383 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.906573 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.906700 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.906904 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.907288 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.907477 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.907603 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.907782 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.907910 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.908097 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.908223 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.908404 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.908529 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.908710 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.908838 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909015 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909149 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909318 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909444 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909638 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909764 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.909954 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.910079 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.910262 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.910397 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.910575 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.910701 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.910913 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911033 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911237 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911378 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911511 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911635 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911824 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.911957 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.912134 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.912260 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.912448 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.912574 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.912755 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.912881 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.913061 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.913194 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.913373 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.913500 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.913686 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.913941 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.914076 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.914277 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.914412 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.914596 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.914722 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.914924 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915061 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915235 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915552 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915688 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915858 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.915985 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.916167 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.916331 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.916556 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.916683 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.916946 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.917370 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.917504 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.917685 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.917819 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.917996 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.918124 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.918311 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.918447 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.918620 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.918747 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.918953 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.919091 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.919282 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.919411 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.919567 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.919695 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.919879 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920013 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920188 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920315 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920498 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920626 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920806 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.920933 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.921125 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.921273 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.921460 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.921588 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.921713 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.921834 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922018 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922155 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922325 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922451 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922635 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922762 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.922963 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.923105 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.923268 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.923394 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.923571 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.923696 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.923877 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924013 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924192 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924318 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924498 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924625 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924810 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.924938 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.925116 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.925242 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.925447 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.925580 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.925759 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.925895 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926065 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926369 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926495 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926695 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926858 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.926996 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.927121 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.927293 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.927429 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.927605 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.927738 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.927912 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928039 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928373 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928539 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928665 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928850 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.928976 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.929162 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.929298 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.929487 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.929619 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.929772 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.929890 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.930079 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.930218 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.930388 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.930521 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.930697 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.930844 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.931025 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.931152 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.931328 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.931467 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.931653 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.931778 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.932456 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.932690 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.932851 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.932976 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.933179 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.933490 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.933626 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.933813 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.934081 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.934271 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.934495 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.934620 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.934758 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.934949 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935075 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935198 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935320 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935489 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935618 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935805 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.935932 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.936122 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.936249 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.936436 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.936566 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.936746 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.936875 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937062 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937195 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937370 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937499 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937690 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937817 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.937998 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.938126 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.938304 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.938432 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.938612 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.938747 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.938945 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.939085 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.939256 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.939383 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.939561 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.939687 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.939870 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940004 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940183 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940308 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940490 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940632 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940795 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.940929 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.941100 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.941228 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.941404 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.941529 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.941712 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.941846 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942016 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942117 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942336 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942442 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942647 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942745 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.942974 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943071 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943279 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943374 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943581 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943677 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943886 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.943982 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.944191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.944288 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.944501 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.944607 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.944803 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.944902 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.945106 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.945203 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.945408 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.945505 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.945712 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.945809 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946019 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946126 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946341 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946445 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946649 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946753 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.946978 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.947091 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.947282 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.947379 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.947631 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.947731 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.947989 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.948233 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.948331 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.948551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.948648 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.948916 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949014 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949225 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949323 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949540 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949637 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949863 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.949958 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.950177 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.950382 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.950522 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.950646 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.950778 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.950941 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.951112 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.951237 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.951424 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.951548 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.951735 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.951945 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952074 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952205 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952329 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952458 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952638 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952774 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.952950 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.953082 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.953261 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.953394 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.953571 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.953709 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.953880 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954013 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954196 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954323 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954510 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954638 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954830 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.954958 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.955142 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.955270 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.955455 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.955583 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.955760 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.955885 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956069 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956193 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956394 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956519 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956674 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956801 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.956982 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.957109 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.957293 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.957426 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.957605 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.957733 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.957916 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.958057 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.958222 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.958348 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.958531 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.958657 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.958861 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959002 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959167 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959293 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959490 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959616 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959797 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.959931 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.960100 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.960226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.960406 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.960534 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.960710 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.960852 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.961020 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.961146 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.961323 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.961449 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.961709 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.961841 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962026 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962144 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962340 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962445 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962661 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962758 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.962980 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.963079 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.963818 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.963966 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.964298 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.964595 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.964729 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.964853 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.964977 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.965101 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.965326 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.965484 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.965643 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.965800 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.965952 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.966115 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.966372 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.966500 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.966687 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.966846 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967035 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967171 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967343 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967468 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967668 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967795 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.967975 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.968114 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.968279 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.968413 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.968582 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.968709 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.968893 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969028 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969193 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969339 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969512 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969640 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969828 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.969955 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.970146 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.970281 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.970457 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.970584 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.970820 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.970952 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.971132 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.971269 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.971449 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.971584 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.971775 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.971906 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.972154 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.972258 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.972480 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.972577 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.972786 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.972883 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.973100 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.973198 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.973409 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.973507 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.973716 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.973818 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974023 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974122 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974337 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974433 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974645 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974755 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.974970 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.975067 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.975294 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.975393 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.975604 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.975751 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.975913 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976039 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976359 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976538 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976675 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976845 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.976972 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.977151 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.977279 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.977476 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.977611 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.977813 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.977952 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.978091 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.978223 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.978395 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.978531 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.978720 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.978952 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.979113 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.979243 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.979424 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.979706 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.979835 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980026 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980154 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980336 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980466 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980646 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980772 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.980953 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.981088 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.981262 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.981397 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.981567 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.981701 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.981872 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982000 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982270 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982409 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982531 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982638 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982829 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.982963 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.983136 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.983273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.983458 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.983586 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.983777 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.983911 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.984081 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.984217 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.984401 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.984528 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.984709 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.984835 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985014 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985149 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985316 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985444 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985620 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.985930 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.986062 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.986239 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.986366 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.986547 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.986687 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.986880 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987008 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987190 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987317 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987498 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987641 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987805 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.987933 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.988132 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.988259 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.988442 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.988578 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.988753 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.988887 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989068 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989194 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989378 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989513 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989680 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989806 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.989983 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.990126 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.990289 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.990415 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.990593 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.990719 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.990917 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991055 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991273 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991408 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991531 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991653 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991819 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.991954 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.992120 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.992255 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.992455 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.992588 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.992731 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.992870 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993034 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993168 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993353 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993480 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993800 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.993964 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.994091 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.994267 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.994979 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.995183 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.995323 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.995582 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.995747 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.996060 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.996209 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.996399 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.996534 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.996714 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.996861 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997041 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997167 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997363 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997490 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997681 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997810 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.997999 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.998125 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.998317 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.998444 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.998635 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.998768 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.998977 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.999105 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.999296 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.999423 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.999609 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.999734 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:58.999926 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.000061 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.000242 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.000368 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.000559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.000679 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.000916 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.001077 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.001235 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.001407 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.001555 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.001723 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.001872 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.002037 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.002228 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.002358 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.002548 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.002678 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.002886 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003025 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003205 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003332 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003524 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003653 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003841 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.003968 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.004157 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.004283 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.004471 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.004599 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.004799 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.004924 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.005094 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.005220 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.005408 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.005545 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.005720 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.005847 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.006034 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.006159 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.006346 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.006485 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.006659 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.006829 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007024 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007159 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007342 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007469 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007650 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007778 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.007966 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.008093 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.008282 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.008408 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.008592 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.008801 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.008928 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.009118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.009254 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.009454 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.009582 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.009782 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.009917 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.010173 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.010892 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.011196 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.011326 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.011520 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.011647 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.011852 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.011981 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.012163 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.012292 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.012471 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.012622 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.012911 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013042 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013163 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013293 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013415 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013537 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013690 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.013817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.014004 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.014309 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.014443 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.014628 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.014763 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.014963 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.015092 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.015300 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.015495 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.015637 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.015842 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.015970 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.016164 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.016291 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.016494 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.016632 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.016821 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.016952 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.017135 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.017269 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.017443 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.017578 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.017748 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.017875 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018051 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018180 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018361 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018487 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018669 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018820 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.018995 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.019123 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.019303 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.019437 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.019609 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.019735 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.019921 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.020135 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.020325 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.020462 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.020588 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.020784 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.020909 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.021108 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.021237 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.021432 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.021568 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.021739 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.021864 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.022042 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.022161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.022348 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.022475 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.022660 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.022811 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023013 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023287 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023414 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023595 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023723 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.023897 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024030 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024201 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024341 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024504 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024629 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024812 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.024939 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.025117 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.025253 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.025426 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.025563 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.025752 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.025882 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.026113 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.026344 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.026488 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.026660 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.026810 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.026997 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.027124 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.027307 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.027434 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.027621 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.027751 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.027928 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028062 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028361 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028540 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028678 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028844 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.028964 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.029159 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.029295 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.029464 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.029619 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.029772 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.030020 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.030148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.030358 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.030493 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.030683 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.030859 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031035 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031168 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031343 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031471 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031665 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031791 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.031970 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.032351 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.032531 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.032658 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.032844 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.032994 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.033176 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.033318 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.033465 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.033591 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.033778 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.033904 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.034090 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.034227 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.034401 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.034527 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.034716 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.034874 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.035055 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.035275 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.035404 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.035606 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.035733 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.035927 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.036065 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.036263 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.036391 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.036571 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.036697 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.036884 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037021 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037186 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037312 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037489 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037616 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037811 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.037949 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.038123 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.038251 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.038436 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.038570 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.038747 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.038924 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.039081 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.039209 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.039391 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.039518 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.039701 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.039830 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040011 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040136 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040319 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040445 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040629 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040756 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.040937 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.041072 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.041245 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.041428 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.041596 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.041723 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.042174 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.042302 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.042494 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.042676 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.042831 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.042932 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.043201 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.043321 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.043492 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.043603 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.043811 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.043899 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.044128 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.044217 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.044446 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.044543 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.044755 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.044841 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.045067 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.045154 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.045377 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.045466 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.045686 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.045773 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046002 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046097 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046335 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046432 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046647 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046734 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.046977 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.047066 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.047284 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.047600 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.047686 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.047924 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048025 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048238 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048325 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048544 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048639 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048852 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.048939 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.049167 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.049255 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.049476 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.049619 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.049782 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.049910 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.050090 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.050234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.050420 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.050558 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.050684 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.050847 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051019 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051146 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051327 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051454 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051636 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051774 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.051945 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052072 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052255 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052362 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052562 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052673 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052870 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.052967 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.053175 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.053285 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.053484 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.053639 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.053809 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.053943 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.054069 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.054193 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.054367 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.054504 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.054671 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.054817 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055008 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055148 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055316 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055444 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055620 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055747 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.055927 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.056063 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.056232 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.056360 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.056536 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.056664 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.056840 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.057502 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.057716 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.057843 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.058037 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.058234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.058361 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.058549 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.058677 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.058887 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059017 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059198 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059326 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059506 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059635 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059813 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.059941 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.060119 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.060248 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.060427 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.060556 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.060759 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.060887 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.061093 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.061222 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.061409 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.061537 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.061732 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.061859 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.062044 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.062180 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.062368 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.062473 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.062678 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.062773 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063017 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063127 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063334 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063421 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063651 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063760 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.063957 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064056 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064268 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064355 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064582 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064678 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064891 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.064986 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.065201 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.065298 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.065511 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.065598 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.065819 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.065908 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.066128 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.066216 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.066444 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.066531 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.066774 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.066902 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.067103 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.067194 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.067427 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.067523 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.067742 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.067828 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.068060 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.068147 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.068374 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.068474 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.068686 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.068773 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069013 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069101 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069322 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069418 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069633 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069720 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.069942 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.070029 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.070249 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.070338 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.070493 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=111938) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 00:43:59.070578 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.070880 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.071193 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.071506 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.071820 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.072135 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.072459 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.073482 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.073940 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.074275 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.074608 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.074980 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.075299 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.075644 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.075976 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.076293 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.076607 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.076925 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:43:59.077169 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=111937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 00:43:59.224621 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO dbcounter [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Registered counter for database nova_api Sep 11 00:43:59.231729 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO dbcounter [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Registered counter for database nova_api Sep 11 00:43:59.246179 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_db.sqlalchemy.engines [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111938) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.252246 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_db.sqlalchemy.engines [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111937) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.252929 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writer thread running {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.259033 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writer thread running {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.310403 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.context [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dae7eaab-b968-41e0-b510-529370888821(cell1) {{(pid=111938) load_cells /opt/stack/nova/nova/context.py:464}} Sep 11 00:43:59.314225 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:43:59.314701 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:43:59.316310 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:43:59.316611 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.context [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dae7eaab-b968-41e0-b510-529370888821(cell1) {{(pid=111937) load_cells /opt/stack/nova/nova/context.py:464}} Sep 11 00:43:59.317032 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:43:59.317480 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:43:59.319502 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:43:59.320447 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_concurrency.lockutils [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111937) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:43:59.320917 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_concurrency.lockutils [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=111937) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:43:59.321488 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO dbcounter [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Registered counter for database nova_cell0 Sep 11 00:43:59.322480 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_concurrency.lockutils [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=111937) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:43:59.324159 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_concurrency.lockutils [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111937) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:43:59.324808 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_concurrency.lockutils [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=111937) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:43:59.326367 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO dbcounter [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Registered counter for database nova_cell1 Sep 11 00:43:59.326598 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_concurrency.lockutils [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=111937) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:43:59.328606 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO dbcounter [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Registered counter for database nova_cell0 Sep 11 00:43:59.332602 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO dbcounter [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Registered counter for database nova_cell1 Sep 11 00:43:59.333500 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_db.sqlalchemy.engines [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111938) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.335228 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_db.sqlalchemy.engines [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111938) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.341378 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writer thread running {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.341927 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_db.sqlalchemy.engines [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111937) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.342688 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_db.sqlalchemy.engines [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111937) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.344268 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writer thread running {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.348804 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writer thread running {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.351445 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writer thread running {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.489815 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO dbcounter [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Registered counter for database nova_cell0 Sep 11 00:43:59.498130 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO dbcounter [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Registered counter for database nova_cell0 Sep 11 00:43:59.498537 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_db.sqlalchemy.engines [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111938) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.503308 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writer thread running {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.506913 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG oslo_db.sqlalchemy.engines [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=111937) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:43:59.512027 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writer thread running {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:43:59.845154 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: WARNING keystonemiddleware.auth_token [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:43:59.845708 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: WARNING keystonemiddleware.auth_token [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:43:59.852486 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: WARNING keystonemiddleware.auth_token [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:43:59.852635 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: WARNING keystonemiddleware.auth_token [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:44:00.195156 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: WARNING keystonemiddleware.auth_token [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:44:00.197338 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: WARNING keystonemiddleware.auth_token [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:44:00.201212 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: WARNING keystonemiddleware.auth_token [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:44:00.203320 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: WARNING keystonemiddleware.auth_token [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:44:00.206503 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cd431bc9c0 pid: 111937 (default app) Sep 11 00:44:00.208609 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:44:00.208867 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cd431bc9c0 pid: 111938 (default app) Sep 11 00:44:00.209840 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f442bdd4-a55f-4e19-9a68-9118df28e85e None None] 10.222.0.27 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001773 Sep 11 00:44:00.210254 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 788 bytes} [Sun Sep 11 00:44:00 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:22.336546 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:22.338400 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002584 Sep 11 00:45:22.338827 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:22 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:22.354874 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_api:SELECT=3 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:22.356362 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_cell1:SELECT=3 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:22.357631 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_cell0:SELECT=3 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:22.359842 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:23.255831 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-04fde9b5-ad58-44d9-aaeb-9ee2b4b5ed78 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:23.289422 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-04fde9b5-ad58-44d9-aaeb-9ee2b4b5ed78 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.944516 Sep 11 00:45:23.290354 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 2/3] 10.222.0.27 () {60 vars in 1292 bytes} [Sun Sep 11 00:45:22 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:25.929402 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:25.930334 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-aefa87dc-ecad-42e5-ae76-6a450b4c7148 None None] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Sep 11 00:45:25.930851 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 2/4] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:26.389647 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f1cdf99a-9126-441e-9853-7f2a4020ddf3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "c1", "vcpus": 1}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:26.437733 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f1cdf99a-9126-441e-9853-7f2a4020ddf3 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.500857 Sep 11 00:45:26.438276 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 3/5] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:25 2022] POST /compute/v2.1/flavors => generated 407 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:26.452790 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_api:SELECT=3 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:26.453585 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_cell1:SELECT=3 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:26.455566 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_cell0:SELECT=3 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:26.456338 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:26.663072 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-3875cb75-c57d-4dbe-8588-1dc942c01d5f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:26.733080 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-3875cb75-c57d-4dbe-8588-1dc942c01d5f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.285562 Sep 11 00:45:26.733675 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 3/6] 10.222.0.27 () {64 vars in 1336 bytes} [Sun Sep 11 00:45:26 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:29.430029 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f1cdf99a-9126-441e-9853-7f2a4020ddf3 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:29.431711 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f1cdf99a-9126-441e-9853-7f2a4020ddf3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002177 Sep 11 00:45:29.432476 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 4/7] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:29 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:30.107921 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-1b12043c-6066-4954-9510-092312b88b8f admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "name": "ds512M", "disk": 5, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 512, "swap": 0, "id": "d1", "os-flavor-access:is_public": true}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:30.151217 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-1b12043c-6066-4954-9510-092312b88b8f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.712997 Sep 11 00:45:30.152396 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 4/8] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:29 2022] POST /compute/v2.1/flavors => generated 404 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:30.165307 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-9306b785-f648-4065-aafe-00242b8bbc2b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:30.207015 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-9306b785-f648-4065-aafe-00242b8bbc2b admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.046092 Sep 11 00:45:30.207907 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 5/9] 10.222.0.27 () {64 vars in 1336 bytes} [Sun Sep 11 00:45:30 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:32.843304 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-1b12043c-6066-4954-9510-092312b88b8f admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:32.844321 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-1b12043c-6066-4954-9510-092312b88b8f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Sep 11 00:45:32.844721 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:33.265615 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-26e23f67-12f1-4eb1-b7a5-898d28dbb714 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "id": "d2", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 1024, "swap": 0, "vcpus": 1, "name": "ds1G", "disk": 10}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:33.285575 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-26e23f67-12f1-4eb1-b7a5-898d28dbb714 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.433662 Sep 11 00:45:33.286127 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 6/11] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:32 2022] POST /compute/v2.1/flavors => generated 404 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:33.299398 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-0a817c29-668b-460d-b592-033c8a382c56 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:33.328219 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-0a817c29-668b-460d-b592-033c8a382c56 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032584 Sep 11 00:45:33.328796 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 6/12] 10.222.0.27 () {64 vars in 1336 bytes} [Sun Sep 11 00:45:33 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:36.054833 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-26e23f67-12f1-4eb1-b7a5-898d28dbb714 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:36.056476 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-26e23f67-12f1-4eb1-b7a5-898d28dbb714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002171 Sep 11 00:45:36.056951 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 7/13] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:36 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:45:36.478434 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-efdbc637-62ea-4a18-b794-4911b27ccf6f admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "disk": 10, "os-flavor-access:is_public": true, "name": "ds2G", "id": "d3", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "vcpus": 2, "swap": 0}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:36.499390 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-efdbc637-62ea-4a18-b794-4911b27ccf6f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.436302 Sep 11 00:45:36.500158 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 7/14] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:36 2022] POST /compute/v2.1/flavors => generated 404 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:36.512450 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-cd9acc35-2df2-4bfb-812d-a51dd83be28c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:36.540370 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-cd9acc35-2df2-4bfb-812d-a51dd83be28c admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031511 Sep 11 00:45:36.542880 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 8/15] 10.222.0.27 () {64 vars in 1336 bytes} [Sun Sep 11 00:45:36 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:39.227037 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-efdbc637-62ea-4a18-b794-4911b27ccf6f admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:39.228061 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-efdbc637-62ea-4a18-b794-4911b27ccf6f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001521 Sep 11 00:45:39.228479 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 8/16] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:39.675141 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-01997d97-4aeb-4130-8929-df182def82ea admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "disk": 20, "swap": 0, "id": "d4", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 4, "os-flavor-access:is_public": true}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:39.696426 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-01997d97-4aeb-4130-8929-df182def82ea admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.461464 Sep 11 00:45:39.697181 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 9/17] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:39 2022] POST /compute/v2.1/flavors => generated 404 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:39.709752 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-e51dcf3f-2543-47b5-b53a-a5edbef9976b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:39.735738 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-e51dcf3f-2543-47b5-b53a-a5edbef9976b admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029841 Sep 11 00:45:39.736320 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 9/18] 10.222.0.27 () {64 vars in 1336 bytes} [Sun Sep 11 00:45:39 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:42.387882 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-01997d97-4aeb-4130-8929-df182def82ea admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:42.388916 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-01997d97-4aeb-4130-8929-df182def82ea admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Sep 11 00:45:42.389291 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 10/19] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:42.836972 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-70dd6a76-5f46-433d-be3a-20dca228fbb9 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:42.862089 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-70dd6a76-5f46-433d-be3a-20dca228fbb9 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.465936 Sep 11 00:45:42.862743 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 10/20] 10.222.0.27 () {60 vars in 1292 bytes} [Sun Sep 11 00:45:42 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:45:45.483561 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-01997d97-4aeb-4130-8929-df182def82ea admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:45.484611 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-01997d97-4aeb-4130-8929-df182def82ea admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001576 Sep 11 00:45:45.485018 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 11/21] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:45.881516 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-2f95c3ea-515f-44f3-812c-db1a1b7b2e31 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 1, "id": "1", "name": "m1.tiny", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 512, "disk": 1}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:45.901207 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-2f95c3ea-515f-44f3-812c-db1a1b7b2e31 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.409258 Sep 11 00:45:45.901789 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 11/22] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:45 2022] POST /compute/v2.1/flavors => generated 402 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 11 00:45:45.914670 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-3853abf3-3bc5-4c5c-9eab-43cd266177c7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:45.941221 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-3853abf3-3bc5-4c5c-9eab-43cd266177c7 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030893 Sep 11 00:45:45.941987 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 12/23] 10.222.0.27 () {64 vars in 1333 bytes} [Sun Sep 11 00:45:45 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:45:48.558932 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-2f95c3ea-515f-44f3-812c-db1a1b7b2e31 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:48.559948 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-2f95c3ea-515f-44f3-812c-db1a1b7b2e31 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Sep 11 00:45:48.560358 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 12/24] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:48.937929 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-e0de2c9b-4f2e-4184-9a1f-b5672b2dd532 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "os-flavor-access:is_public": true, "id": "2", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 1, "swap": 0, "name": "m1.small", "ram": 2048}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:48.960766 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-e0de2c9b-4f2e-4184-9a1f-b5672b2dd532 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.394174 Sep 11 00:45:48.961195 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 13/25] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:48 2022] POST /compute/v2.1/flavors => generated 405 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 11 00:45:48.973699 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-294ef8fa-9adc-4aa3-bda9-60563e0ab422 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:48.997012 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-294ef8fa-9adc-4aa3-bda9-60563e0ab422 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027226 Sep 11 00:45:48.997994 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 13/26] 10.222.0.27 () {64 vars in 1333 bytes} [Sun Sep 11 00:45:48 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:51.667317 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-e0de2c9b-4f2e-4184-9a1f-b5672b2dd532 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:51.669015 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-e0de2c9b-4f2e-4184-9a1f-b5672b2dd532 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002248 Sep 11 00:45:51.669662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 14/27] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:51 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:52.113529 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-b98895df-81cc-43d9-ab50-bea4a5c8fe3b admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "3", "ram": 4096, "disk": 40, "name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:52.138492 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-b98895df-81cc-43d9-ab50-bea4a5c8fe3b admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.462605 Sep 11 00:45:52.139217 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 14/28] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:51 2022] POST /compute/v2.1/flavors => generated 406 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 11 00:45:52.151935 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-e4903ce2-55df-43ce-a08b-9a08eb44deeb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:52.179061 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-e4903ce2-55df-43ce-a08b-9a08eb44deeb admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030866 Sep 11 00:45:52.179994 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 15/29] 10.222.0.27 () {64 vars in 1333 bytes} [Sun Sep 11 00:45:52 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:54.847356 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-b98895df-81cc-43d9-ab50-bea4a5c8fe3b admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:54.848425 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-b98895df-81cc-43d9-ab50-bea4a5c8fe3b admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001549 Sep 11 00:45:54.848833 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 15/30] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:55.296139 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-0cbf8a1c-9d1c-467f-9140-bfe80cc7584f admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "vcpus": 4, "id": "4", "name": "m1.large", "swap": 0, "disk": 80, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 8192, "os-flavor-access:is_public": true}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:55.302634 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:55.317019 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-0cbf8a1c-9d1c-467f-9140-bfe80cc7584f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.461420 Sep 11 00:45:55.318466 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 16/31] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:54 2022] POST /compute/v2.1/flavors => generated 405 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:55.332059 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-2c391587-0f2c-436b-b2f0-20991737230d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:55.357640 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-2c391587-0f2c-436b-b2f0-20991737230d admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030960 Sep 11 00:45:55.367051 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 16/32] 10.222.0.27 () {64 vars in 1333 bytes} [Sun Sep 11 00:45:55 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:45:58.053498 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-0cbf8a1c-9d1c-467f-9140-bfe80cc7584f admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:45:58.054555 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-0cbf8a1c-9d1c-467f-9140-bfe80cc7584f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 Sep 11 00:45:58.055181 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 17/33] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:45:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:45:58.496596 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-a7438f82-755a-4e8b-9eda-fddbbcd34afa admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "5", "os-flavor-access:is_public": true, "ram": 16384, "disk": 160, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 8, "name": "m1.xlarge", "rxtx_factor": 1.0}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:58.505966 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:58.517938 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-a7438f82-755a-4e8b-9eda-fddbbcd34afa admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.455697 Sep 11 00:45:58.518708 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 17/34] 10.222.0.27 () {64 vars in 1283 bytes} [Sun Sep 11 00:45:58 2022] POST /compute/v2.1/flavors => generated 408 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:58.532717 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-9c16328e-d2f3-4515-b2cb-bdbfa2e9d16b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:45:58.562317 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-9c16328e-d2f3-4515-b2cb-bdbfa2e9d16b admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033753 Sep 11 00:45:58.578927 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 18/35] 10.222.0.27 () {64 vars in 1333 bytes} [Sun Sep 11 00:45:58 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:46:15.756444 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-a7438f82-755a-4e8b-9eda-fddbbcd34afa admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:15.757496 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-a7438f82-755a-4e8b-9eda-fddbbcd34afa admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001541 Sep 11 00:46:15.757931 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 18/36] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:46:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:15.768357 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:16.146969 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-7107a3a0-348b-449d-9e39-f2542aec5124 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:16.165748 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-7107a3a0-348b-449d-9e39-f2542aec5124 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.401902 Sep 11 00:46:16.166265 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 19/37] 10.222.0.27 () {60 vars in 1292 bytes} [Sun Sep 11 00:46:15 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:46:17.478104 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-a7438f82-755a-4e8b-9eda-fddbbcd34afa admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:17.479129 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-a7438f82-755a-4e8b-9eda-fddbbcd34afa admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Sep 11 00:46:17.479529 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 19/38] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:18.005437 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-1c60170a-50b5-466a-ad95-cce13b4c3305 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:18.010974 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.wsgi [None req-1c60170a-50b5-466a-ad95-cce13b4c3305 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-1c60170a-50b5-466a-ad95-cce13b4c3305 admin admin] Request method failure captured: Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Accept: application/json Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Accept-Encoding: gzip, deflate Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Connection: keep-alive Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Content-Length: 0 Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Host: 10.222.0.27 Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Openstack-Api-Version: compute 2.92 Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Openstack-System-Scope: None Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: User-Agent: python-novaclient Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Auth-Token: gAAAAABjHS_ZUsRP7Rkv4v6dMtnXEm-THZD_khOoQnQsNGhxrXsaLiSSirJDPYbFUZLxhEnLG7lcpXOowGoH7jr_HqEh1nw7F_j1iwqLfRRaMEJxshgR1MPXOPPDHjKt39DfqmNwtNBqi_gRbqyEkB0O94osqRQw2OGT_TeXaC557fLVufPTmBk Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Domain-Id: None Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Domain-Name: None Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Identity-Status: Confirmed Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Is-Admin-Project: True Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Openstack-Nova-Api-Version: 2.92 Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Domain-Id: default Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Domain-Name: Default Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Name: admin Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Role: member,admin,reader Sep 11 00:46:18.014148 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Roles: member,admin,reader Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/12356f1769944cc8be5c533d062adf65"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_12356f1769944cc8be5c533d062adf65"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant: admin Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant-Name: admin Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User: admin Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Domain-Id: default Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Domain-Name: Default Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Id: 36edaa883d654921ac17e7745c0ff320 Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Name: admin Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: method: > Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: exception: Flavor 42 could not be found. Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: action_args: {'id': '42'} Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:46:18.015817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:18.016852 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.018215 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-1c60170a-50b5-466a-ad95-cce13b4c3305 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=111937) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:46:18.019596 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-1c60170a-50b5-466a-ad95-cce13b4c3305 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.397390 Sep 11 00:46:18.019596 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 20/39] 10.222.0.27 () {60 vars in 1190 bytes} [Sun Sep 11 00:46:17 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 399 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 11 00:46:18.026579 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-5278346a-bcf3-49c6-91e3-9e111b40a380 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:18.050012 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-5278346a-bcf3-49c6-91e3-9e111b40a380 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026965 Sep 11 00:46:18.050547 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 20/40] 10.222.0.27 () {60 vars in 1210 bytes} [Sun Sep 11 00:46:18 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:46:18.062096 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-a33bfe18-ee90-437d-9ce5-ff98a4656c79 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:18.084127 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-a33bfe18-ee90-437d-9ce5-ff98a4656c79 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.025955 Sep 11 00:46:18.084678 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 21/41] 10.222.0.27 () {60 vars in 1210 bytes} [Sun Sep 11 00:46:18 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:46:18.094417 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-eab5cd0f-20ca-440f-8f31-4c4c883eaa21 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:18.100548 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.wsgi [None req-eab5cd0f-20ca-440f-8f31-4c4c883eaa21 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-eab5cd0f-20ca-440f-8f31-4c4c883eaa21 admin admin] Request method failure captured: Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Accept: application/json Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Accept-Encoding: gzip, deflate Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Connection: keep-alive Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Content-Length: 0 Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Host: 10.222.0.27 Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Openstack-Api-Version: compute 2.92 Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Openstack-System-Scope: None Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: User-Agent: python-novaclient Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Auth-Token: gAAAAABjHS_ZUsRP7Rkv4v6dMtnXEm-THZD_khOoQnQsNGhxrXsaLiSSirJDPYbFUZLxhEnLG7lcpXOowGoH7jr_HqEh1nw7F_j1iwqLfRRaMEJxshgR1MPXOPPDHjKt39DfqmNwtNBqi_gRbqyEkB0O94osqRQw2OGT_TeXaC557fLVufPTmBk Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Domain-Id: None Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Domain-Name: None Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Identity-Status: Confirmed Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Is-Admin-Project: True Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Openstack-Nova-Api-Version: 2.92 Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Domain-Id: default Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Domain-Name: Default Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Name: admin Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Role: member,admin,reader Sep 11 00:46:18.104118 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Roles: member,admin,reader Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/12356f1769944cc8be5c533d062adf65"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_12356f1769944cc8be5c533d062adf65"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant: admin Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant-Name: admin Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User: admin Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Domain-Id: default Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Domain-Name: Default Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Id: 36edaa883d654921ac17e7745c0ff320 Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Name: admin Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: method: > Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: exception: Flavor 42 could not be found. Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: action_args: {'id': '42'} Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:46:18.105592 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:18.106759 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:18.108142 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-eab5cd0f-20ca-440f-8f31-4c4c883eaa21 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=111938) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:46:18.109380 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-eab5cd0f-20ca-440f-8f31-4c4c883eaa21 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014702 Sep 11 00:46:18.109380 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 21/42] 10.222.0.27 () {60 vars in 1190 bytes} [Sun Sep 11 00:46:18 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 11 00:46:19.322350 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-a33bfe18-ee90-437d-9ce5-ff98a4656c79 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:19.323400 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-a33bfe18-ee90-437d-9ce5-ff98a4656c79 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Sep 11 00:46:19.323802 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 22/43] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:19.847926 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-acf09a2f-cf32-43d4-b264-3f757aeff0de admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:19.853729 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.wsgi [None req-acf09a2f-cf32-43d4-b264-3f757aeff0de admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-acf09a2f-cf32-43d4-b264-3f757aeff0de admin admin] Request method failure captured: Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Accept: application/json Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Accept-Encoding: gzip, deflate Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Connection: keep-alive Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Content-Length: 0 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Host: 10.222.0.27 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Openstack-Api-Version: compute 2.92 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Openstack-System-Scope: None Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: User-Agent: python-novaclient Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Auth-Token: gAAAAABjHS_blY3zuXfXovhpodwHS2nrBe2ZFRzeMivHUMIUXBaqjujRXyCRK2JzYzFlI59qiPzFx2Rf2TjPOKKWxnJcJqiRpCmdZCrvm9MOAIBxtUNdV2E1QkW5IW7lBW3wU5X-D74sMv6xnkZ48rFMADa_8Bqga7LsiJirKmQSHFnXsxydO38 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Domain-Id: None Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Domain-Name: None Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Identity-Status: Confirmed Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Is-Admin-Project: True Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Openstack-Nova-Api-Version: 2.92 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Domain-Id: default Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Domain-Name: Default Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Name: admin Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Role: member,admin,reader Sep 11 00:46:19.855273 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Roles: member,admin,reader Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/12356f1769944cc8be5c533d062adf65"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_12356f1769944cc8be5c533d062adf65"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant: admin Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant-Name: admin Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User: admin Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Domain-Id: default Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Domain-Name: Default Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Id: 36edaa883d654921ac17e7745c0ff320 Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Name: admin Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: method: > Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: exception: Flavor 84 could not be found. Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: action_args: {'id': '84'} Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:46:19.856757 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:19.859161 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.860357 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-acf09a2f-cf32-43d4-b264-3f757aeff0de admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=111938) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:46:19.861443 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-acf09a2f-cf32-43d4-b264-3f757aeff0de admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.394728 Sep 11 00:46:19.861443 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 22/44] 10.222.0.27 () {60 vars in 1190 bytes} [Sun Sep 11 00:46:19 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 396 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 11 00:46:19.866616 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-21d238c5-c00c-42a0-a3d0-864aa352df3a admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:19.881299 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-21d238c5-c00c-42a0-a3d0-864aa352df3a admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018288 Sep 11 00:46:19.881817 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 23/45] 10.222.0.27 () {60 vars in 1210 bytes} [Sun Sep 11 00:46:19 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:46:19.891213 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-82272c28-2573-4b46-908e-83944ee1b5f6 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:19.907564 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-82272c28-2573-4b46-908e-83944ee1b5f6 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019630 Sep 11 00:46:19.908087 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 23/46] 10.222.0.27 () {60 vars in 1210 bytes} [Sun Sep 11 00:46:19 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:46:19.918130 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-ca22d8e9-548c-404a-985b-c55744d7866c admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:19.923047 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.wsgi [None req-ca22d8e9-548c-404a-985b-c55744d7866c admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-ca22d8e9-548c-404a-985b-c55744d7866c admin admin] Request method failure captured: Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Accept: application/json Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Accept-Encoding: gzip, deflate Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Connection: keep-alive Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Content-Length: 0 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Host: 10.222.0.27 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Openstack-Api-Version: compute 2.92 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Openstack-System-Scope: None Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: User-Agent: python-novaclient Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Auth-Token: gAAAAABjHS_blY3zuXfXovhpodwHS2nrBe2ZFRzeMivHUMIUXBaqjujRXyCRK2JzYzFlI59qiPzFx2Rf2TjPOKKWxnJcJqiRpCmdZCrvm9MOAIBxtUNdV2E1QkW5IW7lBW3wU5X-D74sMv6xnkZ48rFMADa_8Bqga7LsiJirKmQSHFnXsxydO38 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Domain-Id: None Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Domain-Name: None Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Identity-Status: Confirmed Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Is-Admin-Project: True Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Openstack-Nova-Api-Version: 2.92 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Domain-Id: default Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Domain-Name: Default Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Name: admin Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Role: member,admin,reader Sep 11 00:46:19.924551 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Roles: member,admin,reader Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/12356f1769944cc8be5c533d062adf65"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_12356f1769944cc8be5c533d062adf65"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant: admin Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant-Name: admin Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User: admin Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Domain-Id: default Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Domain-Name: Default Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Id: 36edaa883d654921ac17e7745c0ff320 Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Name: admin Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: method: > Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: exception: Flavor 84 could not be found. Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: action_args: {'id': '84'} Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:46:19.926191 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:19.927234 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:19.928662 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-ca22d8e9-548c-404a-985b-c55744d7866c admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=111937) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:46:19.929918 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-ca22d8e9-548c-404a-985b-c55744d7866c admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011327 Sep 11 00:46:19.929918 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 24/47] 10.222.0.27 () {60 vars in 1190 bytes} [Sun Sep 11 00:46:19 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 11 00:46:21.145028 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-82272c28-2573-4b46-908e-83944ee1b5f6 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:21.146182 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-82272c28-2573-4b46-908e-83944ee1b5f6 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001550 Sep 11 00:46:21.146569 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 24/48] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:21.665796 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-59e9d523-5d6a-4e50-86aa-af845ab0e680 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:21.671533 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.wsgi [None req-59e9d523-5d6a-4e50-86aa-af845ab0e680 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-59e9d523-5d6a-4e50-86aa-af845ab0e680 admin admin] Request method failure captured: Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Accept: application/json Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Accept-Encoding: gzip, deflate Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Connection: keep-alive Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Content-Length: 0 Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Host: 10.222.0.27 Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Openstack-Api-Version: compute 2.92 Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Openstack-System-Scope: None Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: User-Agent: python-novaclient Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Auth-Token: gAAAAABjHS_drQbVzXLXOeSQqviOyKR8LPbuWG8h8Al1Q3XvUTymbyvLRRlOuufdgmbJdAzFrWZVh0SVPVbKerfVMf0o5r0ci9GFGpaoVmOdQLDk7TvYK8x5AArSRSAGDlWTy3lFrlYmmloCDldRxkIVOdbTkJXK5mtKV_rwO1xoyXe26wq5cnE Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Domain-Id: None Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Domain-Name: None Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Identity-Status: Confirmed Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Is-Admin-Project: True Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Openstack-Nova-Api-Version: 2.92 Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Domain-Id: default Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Domain-Name: Default Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Project-Name: admin Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Role: member,admin,reader Sep 11 00:46:21.673010 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Roles: member,admin,reader Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/12356f1769944cc8be5c533d062adf65"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_12356f1769944cc8be5c533d062adf65"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant: admin Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-Tenant-Name: admin Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User: admin Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Domain-Id: default Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Domain-Name: Default Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Id: 36edaa883d654921ac17e7745c0ff320 Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: X-User-Name: admin Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: method: > Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: exception: Flavor 451 could not be found. Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: action_args: {'id': '451'} Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:46:21.674559 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:21.675461 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.676597 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-59e9d523-5d6a-4e50-86aa-af845ab0e680 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=111937) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:46:21.678488 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-59e9d523-5d6a-4e50-86aa-af845ab0e680 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.390055 Sep 11 00:46:21.678488 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 25/49] 10.222.0.27 () {60 vars in 1193 bytes} [Sun Sep 11 00:46:21 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 391 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 11 00:46:21.684734 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-9656cf2b-c67e-4bbf-9cb6-c72950f921ec admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:21.699514 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-9656cf2b-c67e-4bbf-9cb6-c72950f921ec admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018343 Sep 11 00:46:21.700035 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 25/50] 10.222.0.27 () {60 vars in 1210 bytes} [Sun Sep 11 00:46:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:46:21.709345 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:21.723813 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.017965 Sep 11 00:46:21.724924 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 26/51] 10.222.0.27 () {60 vars in 1210 bytes} [Sun Sep 11 00:46:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:46:21.734160 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-e7a43f22-ff8f-4dfb-9f21-2766ba15ce6a admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:21.739091 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.wsgi [None req-e7a43f22-ff8f-4dfb-9f21-2766ba15ce6a admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-e7a43f22-ff8f-4dfb-9f21-2766ba15ce6a admin admin] Request method failure captured: Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Accept: application/json Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Accept-Encoding: gzip, deflate Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Connection: keep-alive Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Content-Length: 0 Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Host: 10.222.0.27 Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Openstack-Api-Version: compute 2.92 Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Openstack-System-Scope: None Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: User-Agent: python-novaclient Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Auth-Token: gAAAAABjHS_drQbVzXLXOeSQqviOyKR8LPbuWG8h8Al1Q3XvUTymbyvLRRlOuufdgmbJdAzFrWZVh0SVPVbKerfVMf0o5r0ci9GFGpaoVmOdQLDk7TvYK8x5AArSRSAGDlWTy3lFrlYmmloCDldRxkIVOdbTkJXK5mtKV_rwO1xoyXe26wq5cnE Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Domain-Id: None Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Domain-Name: None Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Identity-Status: Confirmed Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Is-Admin-Project: True Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Openstack-Nova-Api-Version: 2.92 Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Domain-Id: default Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Domain-Name: Default Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Project-Name: admin Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Role: member,admin,reader Sep 11 00:46:21.741369 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Roles: member,admin,reader Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/12356f1769944cc8be5c533d062adf65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/12356f1769944cc8be5c533d062adf65"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_12356f1769944cc8be5c533d062adf65"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant: admin Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant-Id: 12356f1769944cc8be5c533d062adf65 Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-Tenant-Name: admin Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User: admin Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Domain-Id: default Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Domain-Name: Default Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Id: 36edaa883d654921ac17e7745c0ff320 Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: X-User-Name: admin Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: method: > Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: exception: Flavor 451 could not be found. Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: action_args: {'id': '451'} Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:46:21.742923 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:21.743898 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: ERROR nova.api.openstack.wsgi Sep 11 00:46:21.745226 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-e7a43f22-ff8f-4dfb-9f21-2766ba15ce6a admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=111938) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:46:21.746467 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-e7a43f22-ff8f-4dfb-9f21-2766ba15ce6a admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012117 Sep 11 00:46:21.746467 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 26/52] 10.222.0.27 () {60 vars in 1193 bytes} [Sun Sep 11 00:46:21 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 11 00:46:22.971506 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:22.972545 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001502 Sep 11 00:46:22.972936 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 27/53] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:23.494349 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-1d7e2ffc-2130-4dd6-b8db-0447b7e592ca admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:46:23.513110 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-1d7e2ffc-2130-4dd6-b8db-0447b7e592ca admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.402475 Sep 11 00:46:23.513648 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 27/54] 10.222.0.27 () {64 vars in 1235 bytes} [Sun Sep 11 00:46:23 2022] POST /compute/v2.1/flavors => generated 404 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:46:24.741313 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:24.742313 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001424 Sep 11 00:46:24.742700 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 28/55] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:25.377439 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-23e1b8b5-59c8-4b99-80ea-f4877f823c03 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:46:25.397171 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-23e1b8b5-59c8-4b99-80ea-f4877f823c03 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.517242 Sep 11 00:46:25.397765 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 28/56] 10.222.0.27 () {64 vars in 1235 bytes} [Sun Sep 11 00:46:24 2022] POST /compute/v2.1/flavors => generated 405 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 11 00:46:26.624305 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:26.625298 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001413 Sep 11 00:46:26.625690 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 29/57] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:27.155274 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-860952d1-cf95-4f93-a207-fdd28e7be9c7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:46:27.174067 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-860952d1-cf95-4f93-a207-fdd28e7be9c7 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.409764 Sep 11 00:46:27.174608 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 29/58] 10.222.0.27 () {64 vars in 1235 bytes} [Sun Sep 11 00:46:26 2022] POST /compute/v2.1/flavors => generated 407 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:46:28.414252 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:28.415313 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 11 00:46:28.415715 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 30/59] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:28.933107 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-1d7cf2d9-f0a0-43c3-b495-ad0955d67655 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:46:28.951397 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-1d7cf2d9-f0a0-43c3-b495-ad0955d67655 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.397614 Sep 11 00:46:28.951939 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 30/60] 10.222.0.27 () {64 vars in 1235 bytes} [Sun Sep 11 00:46:28 2022] POST /compute/v2.1/flavors => generated 409 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:46:30.178664 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:30.179998 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002582 Sep 11 00:46:30.180538 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 31/61] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:46:30 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:46:30.711678 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-b5e5c7d1-fd8c-4db8-8322-23c89af9e032 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:46:30.732684 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-b5e5c7d1-fd8c-4db8-8322-23c89af9e032 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.403615 Sep 11 00:46:30.733213 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 31/62] 10.222.0.27 () {60 vars in 1202 bytes} [Sun Sep 11 00:46:30 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:47:27.505135 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:47:27.506149 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-f611d229-0845-4bb7-be28-a1b16e199911 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001970 Sep 11 00:47:27.506556 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 32/63] 10.222.0.27 () {54 vars in 931 bytes} [Sun Sep 11 00:47:27 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:47:27.516987 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:27.897109 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-030f62f6-f6ec-471c-9eff-c724850d8cc1 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:47:27.917702 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-030f62f6-f6ec-471c-9eff-c724850d8cc1 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.405115 Sep 11 00:47:27.918238 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 32/64] 10.222.0.27 () {60 vars in 1292 bytes} [Sun Sep 11 00:47:27 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:47:27.939291 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_api:SELECT=14 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:27.943887 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-eb654b5e-10b9-4a40-ac99-6d7f0ceb5334 admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:47:27.951090 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-eb654b5e-10b9-4a40-ac99-6d7f0ceb5334 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013189 Sep 11 00:47:27.951606 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 33/65] 10.222.0.27 () {60 vars in 1283 bytes} [Sun Sep 11 00:47:27 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:47:27.963811 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-b300f344-fea1-49c5-8986-79667344e5bc admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:47:27.971080 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-b300f344-fea1-49c5-8986-79667344e5bc admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010876 Sep 11 00:47:27.971588 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 33/66] 10.222.0.27 () {60 vars in 1286 bytes} [Sun Sep 11 00:47:27 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:47:27.983735 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:47:27.991275 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011060 Sep 11 00:47:27.991787 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 34/67] 10.222.0.27 () {60 vars in 1286 bytes} [Sun Sep 11 00:47:27 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:47:28.002815 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-85307240-1e28-4186-8d7b-dd8021d3dfc8 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:47:28.009984 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-85307240-1e28-4186-8d7b-dd8021d3dfc8 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010643 Sep 11 00:47:28.010487 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 34/68] 10.222.0.27 () {60 vars in 1283 bytes} [Sun Sep 11 00:47:27 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:50:35.464596 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:35.465628 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001442 Sep 11 00:50:35.466003 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 35/69] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:50:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:50:35.605646 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_api:SELECT=6 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:35.763224 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:35.768416 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:35.768882 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:35.769304 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:35.770015 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:35.770459 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:35.770900 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:35.802607 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-3ccdde27-46ff-470a-903a-edb2bca9cb2f admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.198177 Sep 11 00:50:35.803182 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 35/70] 10.222.0.27 () {60 vars in 1193 bytes} [Sun Sep 11 00:50:35 2022] GET /compute/v2.1/os-services => generated 791 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:50:36.893700 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:36.894713 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002117 Sep 11 00:50:36.895151 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 36/71] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:50:36 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:50:37.089262 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:37.092703 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:37.093178 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:37.093607 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:37.094343 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:37.094809 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:37.095227 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:37.115502 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-887b83fd-9cba-4831-8fa3-b622ea186e35 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.082444 Sep 11 00:50:37.116056 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 36/72] 10.222.0.27 () {60 vars in 1193 bytes} [Sun Sep 11 00:50:37 2022] GET /compute/v2.1/os-services => generated 791 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:50:48.620429 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:48.621439 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001415 Sep 11 00:50:48.621831 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 37/73] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:50:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:50:48.765301 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_cell0:SELECT=4 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:48.766019 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_cell1:SELECT=4 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:48.766719 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG dbcounter [-] [111938] Writing DB stats nova_api:SELECT=8 {{(pid=111938) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:48.815909 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:48.819397 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:48.819877 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:48.820340 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:48.821050 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:48.821497 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:48.821911 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:48.847186 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-fa651c22-47b5-4cdd-9fc9-5ee311385e64 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.085656 Sep 11 00:50:48.847633 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 37/74] 10.222.0.27 () {60 vars in 1193 bytes} [Sun Sep 11 00:50:48 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:50:49.925535 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG nova.api.openstack.wsgi [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] Calling method '>' {{(pid=111937) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:49.926507 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: INFO nova.api.openstack.requestlog [None req-7248a8d3-36f7-49ba-9727-41beb0e3474c admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Sep 11 00:50:49.926926 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: [pid: 111937|app: 0|req: 38/75] 10.222.0.27 () {56 vars in 1074 bytes} [Sun Sep 11 00:50:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:50:50.116453 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG nova.api.openstack.wsgi [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Calling method '>' {{(pid=111938) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:50:50.119849 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:50.120318 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:50.120761 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:50.121476 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:50:50.121918 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:50:50.122330 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: DEBUG oslo_concurrency.lockutils [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=111938) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:50:50.143752 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: INFO nova.api.openstack.requestlog [None req-afa789ba-958c-400a-8e2f-6beef41c05c2 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.079518 Sep 11 00:50:50.144391 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: [pid: 111938|app: 0|req: 38/76] 10.222.0.27 () {60 vars in 1193 bytes} [Sun Sep 11 00:50:50 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:51:47.749648 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: Sun Sep 11 00:51:47 2022 - graceful shutdown triggered... Sep 11 00:51:47.749788 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 11 00:51:47.750826 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: Gracefully killing worker 1 (pid: 111937)... Sep 11 00:51:47.751903 dv-cinder-master-3-iscsi devstack@n-api.service[111938]: Gracefully killing worker 2 (pid: 111938)... Sep 11 00:51:47.881504 dv-cinder-master-3-iscsi devstack@n-api.service[111937]: DEBUG dbcounter [-] [111937] Writing DB stats nova_api:SELECT=4 {{(pid=111937) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:48.751337 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: worker 1 buried after 1 seconds Sep 11 00:51:48.751337 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: worker 2 buried after 1 seconds Sep 11 00:51:48.751337 dv-cinder-master-3-iscsi devstack@n-api.service[111936]: goodbye to uWSGI. Sep 11 00:51:48.752901 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 11 00:51:48.753479 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 11 00:51:48.756304 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 11 00:51:48.765650 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 00:51:48 2022] *** Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: nodename: dv-cinder-master-3-iscsi Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: machine: x86_64 Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: clock source: unix Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: pcre jit disabled Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: detected number of CPU cores: 4 Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: current working directory: / Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: detected binary path: /usr/bin/uwsgi-core Sep 11 00:51:48.768619 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: your processes number limit is 47680 Sep 11 00:51:48.769449 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: your memory page size is 4096 bytes Sep 11 00:51:48.769449 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: detected max file descriptor number: 2048 Sep 11 00:51:48.769449 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: lock engine: pthread robust mutexes Sep 11 00:51:48.769449 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: thunder lock: enabled Sep 11 00:51:48.769449 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 11 00:51:48.769449 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 00:51:48.809213 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: Python main interpreter initialized at 0x55b0083979c0 Sep 11 00:51:48.809213 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: python threads support enabled Sep 11 00:51:48.809213 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: your server socket listen backlog is limited to 100 connections Sep 11 00:51:48.809213 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: your mercy for graceful operations on workers is 90 seconds Sep 11 00:51:48.809645 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 00:51:48.809645 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: *** Operational MODE: preforking *** Sep 11 00:51:48.809902 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: *** uWSGI is running in multiple interpreter mode *** Sep 11 00:51:48.809902 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: spawned uWSGI master process (pid: 126968) Sep 11 00:51:48.810106 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 11 00:51:48.810653 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: spawned uWSGI worker 1 (pid: 126979, cores: 1) Sep 11 00:51:48.811169 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: spawned uWSGI worker 2 (pid: 126981, cores: 1) Sep 11 00:51:48.811508 dv-cinder-master-3-iscsi devstack@n-api.service[126968]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 00:51:49.477239 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 11 00:51:49.488320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 11 00:51:50.644536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:51:50.644536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: from cryptography.utils import int_from_bytes Sep 11 00:51:50.645212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:51:50.645212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: from cryptography.utils import int_from_bytes Sep 11 00:51:50.656340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:51:50.656340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: from cryptography.utils import int_from_bytes Sep 11 00:51:50.656827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:51:50.656827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: from cryptography.utils import int_from_bytes Sep 11 00:51:53.051459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 00:51:53.052038 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 00:51:53.052369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 00:51:53.052643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 00:51:53.052903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 00:51:53.053299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.053634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.053963 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.054287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.054622 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.054965 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.055293 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.055618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.055953 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.056211 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.056528 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.057031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.057362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.057680 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.057997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.058327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.058650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.058997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.059320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.059830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.061248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.061248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.061248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.061668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.061888 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.062187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.062519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.062862 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.063765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.064075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.064500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.064824 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.065146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.065466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.065897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.066215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.066525 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.066876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.067204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.067516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.067849 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.068164 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.068490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.068809 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.069118 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.069429 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.069742 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.070064 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.070377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.070682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.070965 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.071274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.071599 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.071924 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.072250 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.072520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.072759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.073079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.073401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.073713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.074021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.074329 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.074636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.074966 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.075275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.075582 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.075901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.076218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.076540 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.076863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.077175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.077496 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.081649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.081649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.081649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.081649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.081649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.081649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.082321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.082670 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.083047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.083368 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.083701 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.084043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.084403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.084726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.084990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.085308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.085629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.085957 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.086277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.086595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.086937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.087249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.087564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.087879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.088192 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.088507 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.088821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.089133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.089443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.089549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 00:51:53.089962 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.090046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 00:51:53.090128 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.090294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 00:51:53.090452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.090664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.091018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.091337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 00:51:53.091447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.091592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 00:51:53.091710 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.091945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.092258 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.092554 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.092680 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.092888 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093013 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093322 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093523 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093833 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.093945 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.094142 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.094246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.094553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.094893 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.095011 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.095283 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.095383 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.095458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.095579 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.095776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.096073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.096251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.096388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.096695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.097010 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.097137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.097327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.097641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.097903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.098024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.098248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.098559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.098759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.098901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.099188 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.099479 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.099589 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.099788 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.100114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.100303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.100439 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.100758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.101041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.101137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.101366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.101691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.101934 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.102038 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.102359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.102706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.103022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.103156 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.103378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.103717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.103894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.104037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.104356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.104637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.104754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.104993 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.105763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.105763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.105931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.106199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.106330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.106512 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.106655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.106905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.107225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.107335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.107552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.107685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.107865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.107984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.108194 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.108452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.108556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.108774 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.108889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.109150 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.109248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.109478 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.109593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.109793 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.110422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.110659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.111067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.111221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.111386 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.111544 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.111699 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.111853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.112012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.112170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.112374 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.112537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.112695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.112850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.113006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.113162 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.113336 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.113690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.113987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.114312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.114475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.114634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.114815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.115003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.115165 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.115322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.115502 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.115668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.115836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.115984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.116381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.116546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.116711 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.116948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.117748 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.117909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.118079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.118233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.118391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.118891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.119065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.119220 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.119369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.119529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.119677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.119822 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.119973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.120126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.120276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.120436 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.120586 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.120755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.120913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.121068 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.121231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.121387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.121534 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.121680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.121825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.121978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.122138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.122293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.122485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.122635 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.122822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.122976 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.123140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.123296 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.123444 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.123590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.123760 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.123984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.124150 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.124331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.124554 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.124715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.124873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.125039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.125186 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.125341 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.125488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.126666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.126858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.127112 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.127273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.127421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.127566 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.127741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.127964 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.128138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.128287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.128441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.128587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.128587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.129007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.129174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.129322 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.129474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.129628 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.129776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.129922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.130081 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.130239 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.130385 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.130385 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.130676 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.130904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.131066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.131224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.131385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.131385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.131663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.131819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.131979 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.132127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.132283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.132430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.132629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.132780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.132941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.133096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.133248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.133395 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.133540 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.133686 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.133919 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.134163 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.134415 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.134586 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.134835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.135003 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.135162 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.135317 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.135465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.135612 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.135757 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.136052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.136218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.136367 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.136781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.136948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.137125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.137352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.137513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.137668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.137815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.137979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.138219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.138386 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.138540 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.138687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.138863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.139161 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.139329 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.139487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.139634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.139787 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.139949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.140096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.140243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.140398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.140553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.140701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.140903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.141055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.141233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.141404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.141553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.141713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.141867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.142014 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.142161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.142317 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.142471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.142677 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.142848 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.143001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.143159 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.143314 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.143480 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.143637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.143792 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.143945 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.144092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.144340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.144502 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.144664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.144836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.145066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.145218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.145376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.145537 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.145694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.145849 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.146012 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.146252 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.146404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.146561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.146997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.147154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.147331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.147554 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.147715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.147876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.148040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.148205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.148435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.148595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.148743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.148902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.149057 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.149298 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.149449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:51:53.149605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.149753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.149909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.150056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.150239 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.150528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.150706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.150896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151046 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151357 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.151962 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.152115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.152261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.152425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.152662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.152881 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153652 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153799 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.153946 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.154099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.154246 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.154492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.154659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.154845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.155012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.155259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.155419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.155567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.155727 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.155883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.156037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.156209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.156434 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.156592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.156793 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.157329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.157544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.157772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.157925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158088 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158953 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.158953 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.159248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.159501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.159653 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.159807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.159954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.160115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.160264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.160672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.160832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.160989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.161137 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.161300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.161455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.161619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.161774 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.161927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.162081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.162240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.162387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.162539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.162685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.162895 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.163776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.163931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.164087 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.164235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.164619 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.164781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.164937 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.165086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.165251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.165502 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.165656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.165803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.165950 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.166103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.166250 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.166511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.166755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.166988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.167433 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.167591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.167766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.168006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.168159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.168313 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.168467 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.168634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.168867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.169036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.169191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.169358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.169530 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.169768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.169929 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.170085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.170232 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.170403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.170636 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.170886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.171069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.171313 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.171481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.171631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.171789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.171936 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.172091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.172238 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.172956 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.173123 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.173483 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.173641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.173804 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.173951 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.174106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.174252 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.174398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.174564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.174881 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.175267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.175418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.175418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.175814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.175972 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176635 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176725 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.176908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177004 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177558 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177771 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.177864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178119 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178311 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178498 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178594 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.178984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179253 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179522 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.179929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180483 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180568 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180652 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180745 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180837 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.180922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181191 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181381 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181470 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181653 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.181922 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182351 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182529 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.182975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183756 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183842 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.183926 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184017 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184364 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184448 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184538 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184824 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.184995 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185086 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185270 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185367 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185782 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.185983 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186071 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186575 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.186980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187678 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.187934 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.188041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.188151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.188151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.188845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.188938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189551 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189642 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189734 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189833 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.189929 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190011 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190084 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190250 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190438 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190526 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.190975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191393 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.191800 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192153 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192637 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.192947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193560 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.193977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.194180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.194283 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.194502 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.194612 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.194816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.194924 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.195127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.195473 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.195576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.195827 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.195937 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196125 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196368 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.196986 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.197088 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.197294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.197386 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.197606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.197720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.197917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198341 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198551 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198651 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.198997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.199195 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.199290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.199505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.199605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.199813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.199908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.200125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.200217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.200433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.200534 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.200737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.200829 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201049 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201457 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201666 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.201989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.202090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.202311 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.202412 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.202631 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.202715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.202959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.203059 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.203270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.203374 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.203586 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.203723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.204042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.204140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.204360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.204460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.204688 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.204772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205730 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.205949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.206055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.206259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.206342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.206568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.206651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.207850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208317 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.208836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.209023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.209309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.209449 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.209630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.209741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.209956 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.210076 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.210292 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.210400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.210643 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.210740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.210994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.211098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.211310 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.211405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.211626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.211717 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.211947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212566 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212872 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.212990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.213191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.213296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.213603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.213718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.213806 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.213909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.214119 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.214223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.214426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.214515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.214738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.214860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.215075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.215180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.215385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.215478 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.215691 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.215786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.216004 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.216126 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.216325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.216451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.216628 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.216721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217256 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217349 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217584 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.217899 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.218006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.218206 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.218358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.218516 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.218673 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.218842 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.219012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.219191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.219855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.220020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.220179 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.220365 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.220610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.220703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.220930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221252 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.221998 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.222226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.222327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.222531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.222630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.222859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.222969 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.223174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.223275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.223482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.223575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.223789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.223893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.224105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.224199 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.224417 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.224525 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.224728 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.224834 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.225961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.226137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.226281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.226446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.226621 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.226754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.226907 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.227121 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.227230 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.227400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.227503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.227713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.227805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228461 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228770 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.228983 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.229076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.229295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.229398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.229599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.229698 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.229903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230311 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.230976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.231178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.231280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.231492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.231594 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.231800 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.231901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.232106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.232209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.232421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.232529 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.232727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.232827 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.233965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.234059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.234272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.234373 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.234583 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.234687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.235631 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.235741 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.235838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.235959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.236146 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.236266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.236459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.236598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.236784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.236931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.237096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.237277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.237564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.237885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.238185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.238295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.238508 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.238601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.239055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.239229 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.239394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.239550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.239720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.239876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240675 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.240985 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.241131 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.241131 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.241496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.241806 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.241917 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.242125 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.242268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.242448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.242614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.242765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.243743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.243874 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.243952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244119 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244282 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244355 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244586 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244868 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.244961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.245182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.245285 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.245521 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.245622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.245833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.245926 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.246149 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.246244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.246490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.246582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.246825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.246923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.247136 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.247240 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.247444 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.247559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.247755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.247850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.248069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.248185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.248383 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.248490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.248693 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.248800 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.249963 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.250216 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.250376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.250551 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.250657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.250898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.251950 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.252184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.252286 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.252497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.252604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.252801 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.252914 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.253112 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.253215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.253422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.253531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.253737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.253832 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.254046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.254149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.254382 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.254484 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.254723 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.254860 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.255102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.255196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.255409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.255510 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.255722 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.255816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.256996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.257109 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.257309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.257404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.257633 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.257746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.257949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.258059 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.258265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.258360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.258581 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.258683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.258924 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259337 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259551 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259857 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.259959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.260163 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.260257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.260470 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.260562 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.260794 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.260903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.261100 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.261190 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.261423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.261515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.261746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.261856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.262064 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.262166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.262386 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.262481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.262868 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.262967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.263186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.263297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.263504 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.263600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.263830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.263942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.264154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.264252 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.264480 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.264603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.264826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.264950 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.265128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.265223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.265439 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.265548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.265755 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.265858 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.266073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.266214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.266500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.266603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.266842 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.266939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.267159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.267267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.267488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.267597 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.267817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.267929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.268135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.268229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.268461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.268555 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.268880 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.268997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.269208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.269352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.269522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.269671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.269844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.270992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.271136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.271326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.271486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.271687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.271788 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.271941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.272064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.272261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.272388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.272576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.272704 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.272886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.273017 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.273863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.273978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274273 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274510 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274584 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.274873 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.275068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.275193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.275378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.275519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.275687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.275996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.276313 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.276625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.276935 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.277243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.277553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.277889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.278178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.278592 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.278939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.279040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.279257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.279385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.279615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.279781 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.279937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.280053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.280242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.280368 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.280584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.280693 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.280880 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.281025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.281205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.281397 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.281496 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.282383 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.282605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.282702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.282832 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.282980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.283072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.283339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.283442 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.283671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.283776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.284024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.284216 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.284371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.284592 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.284702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.284915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.285009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.285236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.285658 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.285846 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.285952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.286161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.286252 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.286472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.286564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.286781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.286912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.287128 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.287221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.287459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.287564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.287770 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.287885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.288085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.288189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.288396 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.288558 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.288716 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.288894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289194 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.289949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.290187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.290377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.290506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.290615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.290844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.290958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.291180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.291274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.291508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.291651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.291754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.291838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.292074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.292209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.292513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.292625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.292757 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.292858 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.293042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.293196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.293367 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.293459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.293688 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.293799 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294624 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294763 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.294967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.295082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.295285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.295377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.295599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.295689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.295917 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.296022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.296224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.296323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.296545 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.296644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.296960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.297106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.297280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.297494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.298243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.298350 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.298602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.298710 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.298955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.299158 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.299286 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.299489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.299595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.299812 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.299911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.300133 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.300229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.300594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.300713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.300810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.300903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.301108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.301202 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.301448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.301614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.301780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.301909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.302155 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.302261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.302397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.302491 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.302711 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.302838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303152 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303680 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.303984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.304165 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.304276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.304490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.304589 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.304806 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.304898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.305167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.305257 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.305469 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.305559 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.305777 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.305876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.306079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.306171 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.306381 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.306473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.306684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.306775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307717 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.307927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.308026 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.308231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.308337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.308539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.308654 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.308840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.309025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.309150 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.309357 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.309462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.309679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.309778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.310000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.310091 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.310324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.310421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.310696 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.310881 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311207 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311552 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311888 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.311994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.312223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.312324 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.312550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.312656 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.312875 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.313963 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.313963 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314153 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314256 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314436 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.314957 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.315712 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.316017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.316285 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.316398 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.316622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.316754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.316966 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.317222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.317330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.317783 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.318146 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.318497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.318852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.319192 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.319307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.319512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.319668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.320026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.320296 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.320495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.320635 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.320834 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.321122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.321224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.321441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.321556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.321769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.321872 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.322096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.322350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.322473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.322678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.322778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.323055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.323342 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.323457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.323547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.323674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.323991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.324101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.324307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.324415 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.324642 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.324746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.324954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325267 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325367 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.325997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.326207 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.326300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.326519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.326614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.326871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327066 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327788 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.327929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=126981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 00:51:53.328201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.328513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.328950 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.329323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.329643 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.329957 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.330290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.330603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.330938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.331266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.331585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.331943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.332263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.332582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.332941 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.333268 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.333828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.334170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.334498 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.334869 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.335211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.335528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.335843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.336156 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.336603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.336928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.337247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.337564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.337881 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.338196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.338512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.338845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.339164 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.339633 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.339955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.340273 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.340588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.340904 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.341217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.341538 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.341885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.342209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.342533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.342877 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.343203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.343618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.343860 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.344174 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.345029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.345354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.345694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.346015 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.346330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.346763 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.348043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.348043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.348043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.348309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.348528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.348860 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.349323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.349651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.349965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.350392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.350748 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.351110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.351443 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.351756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.352515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.352515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.352847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.353018 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.353463 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.353776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.354086 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.354402 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.354713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.355057 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.355371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.355681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.355992 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.356300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.356609 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.356917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.357226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.357534 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.357842 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.358150 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.358456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.358768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.359101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.359409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.359716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.360736 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.361213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.361665 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.362128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.362456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.362831 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.363133 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.363475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.363864 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.364128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.364444 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.364784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.365279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.365605 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.365928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.366246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.366563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.367437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.367437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.367755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.367920 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.368261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.368527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.368832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.369191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.369467 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.369804 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.370122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.370439 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.370755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.371093 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.371409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.371870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.372199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.372518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.372839 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.373160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.373534 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.373827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.374141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.374475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.374828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.375189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.375523 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.375993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.376441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.376767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.377087 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.377408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:51:53.377670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=126979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 00:51:53.394365 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO dbcounter [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Registered counter for database nova_api Sep 11 00:51:53.418033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_db.sqlalchemy.engines [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126981) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.425921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writer thread running {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.444110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO dbcounter [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Registered counter for database nova_api Sep 11 00:51:53.466959 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_db.sqlalchemy.engines [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126979) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.474853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writer thread running {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.486341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.context [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dae7eaab-b968-41e0-b510-529370888821(cell1) {{(pid=126981) load_cells /opt/stack/nova/nova/context.py:464}} Sep 11 00:51:53.490781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:51:53.491341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:51:53.493129 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:51:53.494070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:51:53.494717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:51:53.497045 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:51:53.499238 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO dbcounter [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Registered counter for database nova_cell0 Sep 11 00:51:53.503911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO dbcounter [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Registered counter for database nova_cell1 Sep 11 00:51:53.514149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_db.sqlalchemy.engines [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126981) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.515496 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_db.sqlalchemy.engines [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126981) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.523796 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writer thread running {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.524292 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writer thread running {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.534130 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.context [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dae7eaab-b968-41e0-b510-529370888821(cell1) {{(pid=126979) load_cells /opt/stack/nova/nova/context.py:464}} Sep 11 00:51:53.538441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:51:53.538949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:51:53.540528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:51:53.541257 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:51:53.541720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:51:53.543730 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:51:53.545750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO dbcounter [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Registered counter for database nova_cell0 Sep 11 00:51:53.551030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO dbcounter [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Registered counter for database nova_cell1 Sep 11 00:51:53.558852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_db.sqlalchemy.engines [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126979) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.562207 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_db.sqlalchemy.engines [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126979) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.564135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writer thread running {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.682620 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO dbcounter [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Registered counter for database nova_cell0 Sep 11 00:51:53.691348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_db.sqlalchemy.engines [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126981) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.698350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writer thread running {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.713063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writer thread running {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:53.723251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO dbcounter [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Registered counter for database nova_cell0 Sep 11 00:51:53.730974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_db.sqlalchemy.engines [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=126979) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:51:53.735714 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writer thread running {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:51:54.019957 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: WARNING keystonemiddleware.auth_token [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:51:54.027305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: WARNING keystonemiddleware.auth_token [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:51:54.056597 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: WARNING keystonemiddleware.auth_token [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:51:54.064624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: WARNING keystonemiddleware.auth_token [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:51:54.372414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: WARNING keystonemiddleware.auth_token [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:51:54.378486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: WARNING keystonemiddleware.auth_token [None req-623804d6-8072-456d-808b-c6f7379ebb08 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:51:54.383777 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b0083979c0 pid: 126981 (default app) Sep 11 00:51:54.413377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: WARNING keystonemiddleware.auth_token [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 11 00:51:54.419376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: WARNING keystonemiddleware.auth_token [None req-123c6df2-5f86-4be2-9c82-5b0af14df996 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 11 00:51:54.425354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b0083979c0 pid: 126979 (default app) Sep 11 00:52:33.251209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c5c9b235-33fe-4e2b-90dd-c57a5c54b485 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01bc65f70>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:33.326042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:33.326042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:33.326042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:33.326042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:34.495181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c5c9b235-33fe-4e2b-90dd-c57a5c54b485 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/874a813a-a1a3-4d26-bbeb-93ca78e6ea2b" status: 200 len: 351 microversion: 2.1 time: 1.882013 Sep 11 00:52:34.497695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 1/1] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:52:32 2022] GET /compute/v2.1/os-volumes/874a813a-a1a3-4d26-bbeb-93ca78e6ea2b => generated 351 bytes in 1886 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:37.204694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-74245592-f835-4e31-85db-f71ddb4c42b6 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01bc64f70>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:37.241606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:37.243617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:37.245090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:37.246135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:38.240462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-74245592-f835-4e31-85db-f71ddb4c42b6 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/e3da3d4b-ba71-482c-b300-7b95e4d1a612" status: 200 len: 351 microversion: 2.1 time: 1.620014 Sep 11 00:52:38.252273 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 1/2] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:52:36 2022] GET /compute/v2.1/os-volumes/e3da3d4b-ba71-482c-b300-7b95e4d1a612 => generated 351 bytes in 1634 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:40.018939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0f3013d5-4e19-4ecb-8e64-fdef84f451fd tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba6b3a0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:40.133051 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0f3013d5-4e19-4ecb-8e64-fdef84f451fd tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/5b0713f3-7f2a-45f0-b3b0-655fdc79d6b8" status: 200 len: 351 microversion: 2.1 time: 0.119603 Sep 11 00:52:40.134127 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:52:40 2022] GET /compute/v2.1/os-volumes/5b0713f3-7f2a-45f0-b3b0-655fdc79d6b8 => generated 351 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:40.152954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f3fe46ed-7bec-486a-8e58-385d52f7f712 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba75310>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:40.297205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f3fe46ed-7bec-486a-8e58-385d52f7f712 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.149252 Sep 11 00:52:40.298007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1134 bytes} [Sun Sep 11 00:52:40 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:40.316641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-00663c75-3069-4116-8442-32af1dccf7ff tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba6b550>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:40.594899 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-00663c75-3069-4116-8442-32af1dccf7ff tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.282481 Sep 11 00:52:40.606872 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 3/5] 10.222.0.27 () {58 vars in 1149 bytes} [Sun Sep 11 00:52:40 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:40.610453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cec05bfd-6641-452e-945c-ee8542216b49 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba790d0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:40.844305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cec05bfd-6641-452e-945c-ee8542216b49 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.237563 Sep 11 00:52:40.851263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 3/6] 10.222.0.27 () {58 vars in 1134 bytes} [Sun Sep 11 00:52:40 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:40.860889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-68532a80-8ed9-4e01-ae64-5a88c7d52177 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba6bdc0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:40.982036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-68532a80-8ed9-4e01-ae64-5a88c7d52177 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.126141 Sep 11 00:52:40.993141 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1167 bytes} [Sun Sep 11 00:52:40 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:41.000855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-38ef8a18-8809-4582-a57e-b62829b63ef2 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba798b0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:41.118717 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-38ef8a18-8809-4582-a57e-b62829b63ef2 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.125279 Sep 11 00:52:41.132307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1170 bytes} [Sun Sep 11 00:52:40 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:41.144658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-dff78bb5-2887-4c2f-8381-d131a53e63a4 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba99700>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:41.381933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-dff78bb5-2887-4c2f-8381-d131a53e63a4 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.250534 Sep 11 00:52:41.383072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 5/9] 10.222.0.27 () {58 vars in 1155 bytes} [Sun Sep 11 00:52:41 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:41.399318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-82f83941-a64c-405c-bf6b-1d247ebed320 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba95310>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:41.626359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-82f83941-a64c-405c-bf6b-1d247ebed320 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.231254 Sep 11 00:52:41.627337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 5/10] 10.222.0.27 () {58 vars in 1188 bytes} [Sun Sep 11 00:52:41 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:52:41.657344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4708b392-a986-49bb-b5e6-1cc23a16ee26 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] Calling method '.version_select at 0x7fc01ba998b0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:42.005862 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4708b392-a986-49bb-b5e6-1cc23a16ee26 tempest-VolumesTestJSON-1587953580 tempest-VolumesTestJSON-1587953580-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.366011 Sep 11 00:52:42.006451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 6/11] 10.222.0.27 () {58 vars in 1155 bytes} [Sun Sep 11 00:52:41 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:42.741207 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1745102154", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:52:42.944547 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] validate_networks() for [('4d017d49-42d2-4f66-ab1e-6d7b9f9e874f', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:52:45.142776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:52:45.143580 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:52:45.145265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:52:45.145993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:52:45.146529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:52:45.180784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:45.181566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:45.182229 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:45.183354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:45.184136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:45.184789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:45.243767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for project 6a58aefc25e141ca93821a80859875e8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:52:45.254458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for user 5df6979effd84ece8d75b631ba7da196 and project 6a58aefc25e141ca93821a80859875e8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:52:45.271757 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:52:45.279315 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:52:45.426071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-70f49bdf-85fb-4c0a-bc98-373aa7470a00 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.219507 Sep 11 00:52:45.429148 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 6/12] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:52:42 2022] POST /compute/v2.1/servers => generated 383 bytes in 3223 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:52:45.440625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d83d648f-e8e8-450c-9155-90f4ae5c8021 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:45.441400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d83d648f-e8e8-450c-9155-90f4ae5c8021 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:45.638007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d83d648f-e8e8-450c-9155-90f4ae5c8021 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:52:45.668500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d83d648f-e8e8-450c-9155-90f4ae5c8021 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1222 microversion: 2.1 time: 0.232908 Sep 11 00:52:45.671335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:45 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1222 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:46.270128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-951868800", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:52:46.378739 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] validate_networks() for [('4d017d49-42d2-4f66-ab1e-6d7b9f9e874f', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:52:48.432579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:52:48.433168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:52:48.433482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:52:48.434667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:52:48.435275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:52:48.516645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:48.517217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:48.519276 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:48.520173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:48.520544 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:48.520971 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:48.606614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for project 6a58aefc25e141ca93821a80859875e8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:52:48.626521 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for user 5df6979effd84ece8d75b631ba7da196 and project 6a58aefc25e141ca93821a80859875e8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:52:48.680741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:52:48.694498 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:52:48.840560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f404388c-9476-47db-8cd1-015fac106c3c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.571573 Sep 11 00:52:48.844205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 7/14] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:52:46 2022] POST /compute/v2.1/servers => generated 383 bytes in 2580 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:52:48.857101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-63db4929-f4b7-4f16-b2b0-9cf3ff52c50d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:48.858982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-63db4929-f4b7-4f16-b2b0-9cf3ff52c50d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:49.221391 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-63db4929-f4b7-4f16-b2b0-9cf3ff52c50d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:52:49.233804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-63db4929-f4b7-4f16-b2b0-9cf3ff52c50d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1221 microversion: 2.1 time: 0.380930 Sep 11 00:52:49.236099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 8/15] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:48 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1221 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:49.911218 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-786cc726-909f-44a2-ab05-f36f071f7edf tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'create', calling method: .version_select at 0x7fc01b128670>, body: {"keypair": {"name": "tempest-keypair-1743888382", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEo9nBaOfkCbbnlF01HntNKt5NCR4ulKVkhWZHK8bRNuEgi1jhNYqznvkQxmN3En1ueneLK6OWZoRnPvrhRodUjzIDOc/1SsYz9zE6DkILyHN3JrITg9AyTtI6xwD5tb0Q=="}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:52:50.004808 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-786cc726-909f-44a2-ab05-f36f071f7edf tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for project fd9221b2e81a46f69864e02ec4d42672. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:52:50.013896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-786cc726-909f-44a2-ab05-f36f071f7edf tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for user 73a3d3c48913474ab8fa8fedd1bc7d03 and project fd9221b2e81a46f69864e02ec4d42672. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:52:50.111032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-786cc726-909f-44a2-ab05-f36f071f7edf tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.479283 Sep 11 00:52:50.112272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 8/16] 10.222.0.27 () {62 vars in 1200 bytes} [Sun Sep 11 00:52:49 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 481 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 11 00:52:50.258216 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2c3df179-c212-4001-aad1-88625bee3ba6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:50.258952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2c3df179-c212-4001-aad1-88625bee3ba6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:50.429597 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2c3df179-c212-4001-aad1-88625bee3ba6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:52:50.438707 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2c3df179-c212-4001-aad1-88625bee3ba6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1221 microversion: 2.1 time: 0.185169 Sep 11 00:52:50.462502 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 9/17] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:50 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1221 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:51.478768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:51.480203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:51.509155 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:51.509887 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:51.510553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:51.511927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:51.515672 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:51.517048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:53.315880 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:52:53.404158 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-15000aab-4d78-477c-bda6-80ccc4b675af tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1273 microversion: 2.1 time: 1.930738 Sep 11 00:52:53.415797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 9/18] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:51 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1273 bytes in 1943 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:54.033941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-290782752", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "networks": [{"uuid": "a5e22fa6-8ae2-441b-a3f1-0f5e0f56ff3a"}]}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:52:54.411230 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] validate_networks() for [('a5e22fa6-8ae2-441b-a3f1-0f5e0f56ff3a', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:52:54.442136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:54.442771 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:54.464314 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:54.466838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:54.470500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:54.471499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:54.475343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:54.478357 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:55.290659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:52:55.304075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d419c858-cfc2-4d51-83f6-235a4e0bd20a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1279 microversion: 2.1 time: 0.867875 Sep 11 00:52:55.305344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 10/19] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:54 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1279 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:52:56.355410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:56.356170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:56.384332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:56.388360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:56.390242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:56.395927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:56.400585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:56.402400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:57.058103 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:52:57.067293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-46b2d957-1471-48a3-935c-29f2b40d23e7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1279 microversion: 2.1 time: 0.719376 Sep 11 00:52:57.072134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 11/20] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:56 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1279 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:52:58.095332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:52:58.096874 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:52:58.115809 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:58.118911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:58.119651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:58.120565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:52:58.134914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:52:58.137940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:52:58.555481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=4 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:00.046316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=16 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:00.252440 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:00.272200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6631fcff-7b3a-491c-a99e-7b61e6b640c6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 2.181500 Sep 11 00:53:00.287165 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 12/21] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:52:58 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 2197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:00.321431 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-896596510", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3c897342-0f04-45f8-a9ef-0bc52e6c6ab5"}], "security_groups": [{"name": "tempest-securitygroup--494861835"}], "key_name": "tempest-keypair-1743888382", "user_data": "}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:00.436664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:00.704416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:53:00.705387 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:53:00.706040 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:53:00.706992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:53:00.707893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:53:00.770922 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:00.773664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:00.775161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:00.777085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:00.778667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:00.780817 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:00.853648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Getting quotas for project 832fdbe87ca847f094c6cc8a1fc43a59. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:53:00.863912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Getting quotas for user 8b1fc2c958f045538fb15d7d1d4a408a and project 832fdbe87ca847f094c6cc8a1fc43a59. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:53:00.885468 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:53:00.891683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:53:01.150109 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3e4379cd-b557-4aa2-bbf7-bf57327049c3 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 8.337763 Sep 11 00:53:01.152571 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 10/22] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:52:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 8341 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 11 00:53:01.307787 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] validate_networks() for [('3c897342-0f04-45f8-a9ef-0bc52e6c6ab5', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:53:02.028480 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ae9ae130-386b-44b9-b9e6-58a78769bc4a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: .version_select at 0x7fc01b952280>, body: {"keypair": {"name": "tempest-keypair-1190363958", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOYQENsLvSYpka+Lua+FT+0+mzgHZgrFyxzdioAhE3X9sIzFLknCf8YKCn6HQNh/Fd0OeUaJZjZpm0jlhfv+lsY5j2MUpU0Uz81ljDb161SWPM6tGX4v0kKYps9cqIe27w=="}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:02.098082 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-ae9ae130-386b-44b9-b9e6-58a78769bc4a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:53:02.113341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-ae9ae130-386b-44b9-b9e6-58a78769bc4a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:53:02.196583 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ae9ae130-386b-44b9-b9e6-58a78769bc4a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.023735 Sep 11 00:53:02.210528 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 11/23] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:53:01 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1057 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:02.217669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bbaf4d70-1ddb-4c28-aef7-5562669d7290 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:02.218601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bbaf4d70-1ddb-4c28-aef7-5562669d7290 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:02.410492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bbaf4d70-1ddb-4c28-aef7-5562669d7290 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:02.423975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bbaf4d70-1ddb-4c28-aef7-5562669d7290 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1214 microversion: 2.1 time: 0.210298 Sep 11 00:53:02.443460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 12/24] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:02 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1214 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:02.460010 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:02.460539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:02.499076 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:02.499978 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:02.505522 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:02.509988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:02.511894 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:02.517311 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:04.685130 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:04.700775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:53:04.703063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:53:04.704124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:53:04.704841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:53:04.705443 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:53:04.710520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-784635a4-7a78-4c62-bccb-f1063f379515 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 2.265227 Sep 11 00:53:04.739932 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:04.741205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 13/25] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:02 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 2296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:04.742375 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:04.745007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:04.749446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:04.752357 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:04.753747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:04.760268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:04.761322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:04.787227 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:04.787970 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:04.788598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:04.790507 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:04.796273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:04.796273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:04.814170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for project fd9221b2e81a46f69864e02ec4d42672. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:53:04.825246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for user 73a3d3c48913474ab8fa8fedd1bc7d03 and project fd9221b2e81a46f69864e02ec4d42672. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:53:04.872753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:53:04.886645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:53:05.048379 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1f1890b4-1c45-4339-9822-277fca3b654e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 4.758679 Sep 11 00:53:05.053428 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 13/26] 10.222.0.27 () {62 vars in 1188 bytes} [Sun Sep 11 00:53:00 2022] POST /compute/v2.1/servers => generated 401 bytes in 4766 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 11 00:53:05.075536 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8e39b836-b69b-4b7b-a89d-a33e305e87f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:05.090703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8e39b836-b69b-4b7b-a89d-a33e305e87f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:05.195596 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:05.210209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-dd03eb3c-e3f1-4f2a-8125-fde73c911f86 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1272 microversion: 2.1 time: 0.467106 Sep 11 00:53:05.213665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 14/27] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:04 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1272 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:05.275557 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8e39b836-b69b-4b7b-a89d-a33e305e87f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:05.380719 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8e39b836-b69b-4b7b-a89d-a33e305e87f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:05.400814 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8e39b836-b69b-4b7b-a89d-a33e305e87f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:05.412391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8e39b836-b69b-4b7b-a89d-a33e305e87f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1281 microversion: 2.20 time: 0.340013 Sep 11 00:53:05.446705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 14/28] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:05 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1281 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:05.747200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:05.751091 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:05.795606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:05.798313 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:05.799886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:05.807797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:05.809236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:05.812721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:06.240141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:06.241122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:06.252938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:06.257824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:06.265590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:06.266999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:06.267640 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:06.269706 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:06.688948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:06.705200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-54936d89-7925-41c0-859d-fc65ebb94603 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1272 microversion: 2.1 time: 0.469806 Sep 11 00:53:06.715812 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 15/29] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:06 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1272 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:06.731100 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:06.734582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:06.760039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:06.764720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:06.766084 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:06.768341 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:06.772524 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:06.778811 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.157406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:07.164403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:07.171623 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:07.178399 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:07.182481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3c37c2d8-045d-4f1a-a047-5a6bc232e3fb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1277 microversion: 2.20 time: 0.463131 Sep 11 00:53:07.184426 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fc33ebfb-a09e-464e-969c-00c738c97003 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 1.447400 Sep 11 00:53:07.184974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 16/30] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:06 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1277 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:07.195702 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 15/31] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:05 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 1460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:07.318828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62f2702d-714f-47f4-87e5-25f0283ee9a5", "tag": "d093dbf0-edb8-4356-a6c6-97c0aa32850c"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:07.368258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=10 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:07.376445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.379001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.379686 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.434816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=43 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:07.452241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9acff3d4-2dd1-4809-bc64-2dfc7829bfc6", "tag": "1fc55f70-880d-4007-99be-49ec52f1b61e"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:07.452943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Creating event network-changed:d093dbf0-edb8-4356-a6c6-97c0aa32850c for instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 on hv-cinder-maste Sep 11 00:53:07.453823 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=126979) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 11 00:53:07.455277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-adcff9b3-112f-49f0-9b5b-647783332cb0 req-f6f7b98f-0b9a-436c-b277-3c8fd39bc8b4 service nova] Acquired lock "compute-rpcapi-router" {{(pid=126979) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 11 00:53:07.459632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.460308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.462582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.464573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.468380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.469600 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.499168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.499677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.500127 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.511429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.compute.rpcapi [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 11 00:53:07.512559 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] Releasing lock "compute-rpcapi-router" {{(pid=126979) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 11 00:53:07.540901 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1045f8f6-03d1-438a-8a85-4bde7bcaf879 None None] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.353249 Sep 11 00:53:07.542191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 17/32] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:07.569696 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Creating event network-changed:1fc55f70-880d-4007-99be-49ec52f1b61e for instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 on hv-cinder-maste Sep 11 00:53:07.570231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Acquiring lock "compute-rpcapi-router" {{(pid=126981) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 11 00:53:07.570630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bac1d426-f899-4c9c-9849-52bec2a5c3a5 req-1796a371-d71e-4fbf-8784-eb6f133808fe service nova] Acquired lock "compute-rpcapi-router" {{(pid=126981) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 11 00:53:07.600246 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.603297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.608199 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.611595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.615511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.618685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.677268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.compute.rpcapi [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 11 00:53:07.680955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] Releasing lock "compute-rpcapi-router" {{(pid=126981) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 11 00:53:07.717385 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-78c64e93-e4bc-4391-b08f-d3bc404882b0 None None] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.400550 Sep 11 00:53:07.720165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 16/33] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:53:07.743647 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:07.744357 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:07.761932 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.762756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.763578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:07.764496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:07.764961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:07.765390 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:08.043302 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:08.071228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fe17bc08-e493-4d0e-8b5f-72a38791d7a4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1272 microversion: 2.1 time: 0.331873 Sep 11 00:53:08.074406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 18/34] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:07 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1272 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:08.222738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:08.224151 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:08.224696 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:08.227929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:08.245745 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:08.247568 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:08.253403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:08.254096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:08.257463 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:08.260015 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:08.260648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:08.261766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:08.261992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:08.265362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:08.267251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:08.270896 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:08.533627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:08.555545 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:08.557750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:08.559290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5b8d955f-48dc-4ceb-8393-0325b12720f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1341 microversion: 2.20 time: 0.348855 Sep 11 00:53:08.560429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 19/35] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:08 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1341 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:08.659463 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:08.669487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bd913aa8-3262-4927-a0dc-8e22874c0d79 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 0.460294 Sep 11 00:53:08.674486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 17/36] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:08 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:09.106276 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:09.107845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:09.127181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:09.128520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:09.130009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:09.132432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:09.136759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:09.137566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:09.585191 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:09.585814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:09.598324 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:09.599702 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:09.600314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:09.601320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:09.601913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:09.602484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:09.995057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:10.014555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:10.016075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:10.017200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c9e7d824-4c59-40cc-bee9-de833d8fdb4a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1339 microversion: 2.20 time: 0.435954 Sep 11 00:53:10.024576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 18/37] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:09 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1339 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:10.035864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:10.036704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:10.071582 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:10.072433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:10.072970 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:10.073950 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:10.074565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:10.075174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:10.201519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:10.212882 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6df00e56-ac5d-4e51-9306-e79d96aff73e tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 1.117522 Sep 11 00:53:10.215823 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 20/38] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:09 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 1121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:10.682934 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:10.702454 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-698014868", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e5ef326a-1def-4c9c-ad55-20a6d9fdbde1"}], "security_groups": [{"name": "tempest-securitygroup--1859556586"}], "key_name": "tempest-keypair-1190363958", "user_data": "}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:10.704308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c90579d7-0a65-44d6-a4c0-fbf89f3d9b22 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 0.676530 Sep 11 00:53:10.707326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 19/39] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:10 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:10.845465 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] validate_networks() for [('e5ef326a-1def-4c9c-ad55-20a6d9fdbde1', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:53:11.054377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:11.055178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:11.067838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:11.069085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:11.071210 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:11.073808 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:11.074366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:11.074817 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:11.185395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:11.195267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:11.197295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:11.198670 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-97642a2c-3c5d-4880-824c-d2a9bcbedd9a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1339 microversion: 2.20 time: 0.150302 Sep 11 00:53:11.199322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 20/40] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:11 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1339 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:11.242247 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:11.243987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:11.252669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:11.253574 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:11.254093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:11.254966 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:11.255655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:11.256187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:11.763073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:11.775561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a99daba3-f680-4c46-84e1-9bac0c5d9b13 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.537611 Sep 11 00:53:11.791465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 21/41] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:11 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:11.802431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:11.803474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:11.813561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:11.818234 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:11.820677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:11.821598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:11.822231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:11.824267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.417411 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=12 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:12.570632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:53:12.571981 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:53:12.573142 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:53:12.573881 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:53:12.574441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:53:12.597724 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.598873 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.599466 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.600927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.602299 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.605666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.651701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:53:12.661978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:53:12.701071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:53:12.704139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:53:12.800532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1ad82eea-4f88-4460-97de-0df2f47c5e3e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.102342 Sep 11 00:53:12.801848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 21/42] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:53:10 2022] POST /compute/v2.1/servers => generated 402 bytes in 2104 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:53:12.806673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-f0cd138c-481e-4c7f-b176-cce883c473c8 req-d619935a-6369-498b-8e6e-cdb17c82f0b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d334d7b3-7a92-42bd-902a-39728c315ee9", "tag": "1a22d551-a8f3-4dd7-bcbb-3d16039c58ef"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:12.818213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f0cd138c-481e-4c7f-b176-cce883c473c8 req-d619935a-6369-498b-8e6e-cdb17c82f0b1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.819201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f0cd138c-481e-4c7f-b176-cce883c473c8 req-d619935a-6369-498b-8e6e-cdb17c82f0b1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.819850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f0cd138c-481e-4c7f-b176-cce883c473c8 req-d619935a-6369-498b-8e6e-cdb17c82f0b1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.835037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:12.845584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f5dcf6ce-7012-4689-a0fa-942b2bd7093a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 1.052657 Sep 11 00:53:12.846204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 22/43] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:11 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 1054 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:12.864280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:12.865350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:12.873135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.873777 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.874222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.874992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.875573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.876028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.880623 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-f0cd138c-481e-4c7f-b176-cce883c473c8 req-d619935a-6369-498b-8e6e-cdb17c82f0b1 service nova] Creating event network-changed:1a22d551-a8f3-4dd7-bcbb-3d16039c58ef for instance d334d7b3-7a92-42bd-902a-39728c315ee9 on hv-cinder-maste Sep 11 00:53:12.893094 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-f0cd138c-481e-4c7f-b176-cce883c473c8 req-d619935a-6369-498b-8e6e-cdb17c82f0b1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089558 Sep 11 00:53:12.895656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 22/44] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:12.906758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:12.907874 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:12.930171 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.931811 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.932263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:12.933171 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:12.934074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:12.934504 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:13.411206 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:13.426259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:13.428160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:13.430928 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d0470f67-ecf1-4cb1-ae1b-c4e980244fdc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.583659 Sep 11 00:53:13.448810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 23/45] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:12 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:13.459558 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-63f82d31-98fa-42b2-8d28-6026262217ed tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:13.461628 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:13.463820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-63f82d31-98fa-42b2-8d28-6026262217ed tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:13.491569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1630b859-dc5c-43e5-aa05-dd76f7f4911d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.591200 Sep 11 00:53:13.493544 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 23/46] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:12 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:13.592958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-63f82d31-98fa-42b2-8d28-6026262217ed tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:13.608028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-63f82d31-98fa-42b2-8d28-6026262217ed tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1241 microversion: 2.1 time: 0.157057 Sep 11 00:53:13.611010 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 24/47] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:13 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1241 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:13.877173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:13.878141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:13.894530 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:13.895144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:13.898615 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:13.901159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:13.905347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:13.906014 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:14.428154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:14.441037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b7a52180-f86a-499e-a3a8-1767593693f1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 0.568527 Sep 11 00:53:14.441821 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 24/48] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:13 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:14.462486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:14.463740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:14.476823 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:14.480311 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:14.480857 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:14.487713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:14.488595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:14.489043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:14.520373 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:14.521225 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:14.533959 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:14.534456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:14.535369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:14.536238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:14.536856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:14.537434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:15.117278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:15.136047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:15.139404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:15.143678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3791923e-7b4a-4b3c-9370-8b711c5b043a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.684989 Sep 11 00:53:15.147845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 25/49] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:14 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:15.159287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:15.160576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:15.174531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:15.175854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:15.176446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:15.176593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:15.177811 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:15.178669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:15.179264 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:15.190237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b6524807-5056-4abd-84b6-74e9deb3de6a tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.671675 Sep 11 00:53:15.194053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 25/50] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:14 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:15.308140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:15.322388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e5e552f6-dc41-4d41-85ea-2e0e2d56cf02 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1293 microversion: 2.1 time: 0.172153 Sep 11 00:53:15.324166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 26/51] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:15 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1293 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:15.339635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-7273c2eb-e970-42ca-ab2b-dc710ffa7345 req-6340b4ba-5250-449e-acab-1d5432ddbf39 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:15.351501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7273c2eb-e970-42ca-ab2b-dc710ffa7345 req-6340b4ba-5250-449e-acab-1d5432ddbf39 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:15.352207 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7273c2eb-e970-42ca-ab2b-dc710ffa7345 req-6340b4ba-5250-449e-acab-1d5432ddbf39 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:15.352649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7273c2eb-e970-42ca-ab2b-dc710ffa7345 req-6340b4ba-5250-449e-acab-1d5432ddbf39 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:15.358446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-c06d14ea-92db-4825-91d4-a3b22c6b233f req-f84168c2-9297-4a44-999f-2c641252956e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62f2702d-714f-47f4-87e5-25f0283ee9a5", "name": "network-vif-plugged", "status": "completed", "tag": "d093dbf0-edb8-4356-a6c6-97c0aa32850c"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:15.369726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c06d14ea-92db-4825-91d4-a3b22c6b233f req-f84168c2-9297-4a44-999f-2c641252956e service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:15.370369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c06d14ea-92db-4825-91d4-a3b22c6b233f req-f84168c2-9297-4a44-999f-2c641252956e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:15.370883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c06d14ea-92db-4825-91d4-a3b22c6b233f req-f84168c2-9297-4a44-999f-2c641252956e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:15.381079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-7273c2eb-e970-42ca-ab2b-dc710ffa7345 req-6340b4ba-5250-449e-acab-1d5432ddbf39 service nova] Creating event network-changed:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:53:15.386583 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-7273c2eb-e970-42ca-ab2b-dc710ffa7345 req-6340b4ba-5250-449e-acab-1d5432ddbf39 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058289 Sep 11 00:53:15.387281 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 27/52] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:15.389914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-c06d14ea-92db-4825-91d4-a3b22c6b233f req-f84168c2-9297-4a44-999f-2c641252956e service nova] Creating event network-vif-plugged:d093dbf0-edb8-4356-a6c6-97c0aa32850c for instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 on hv-cinder-maste Sep 11 00:53:15.396506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-c06d14ea-92db-4825-91d4-a3b22c6b233f req-f84168c2-9297-4a44-999f-2c641252956e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.200864 Sep 11 00:53:15.397380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 26/53] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:15.477154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:15.478045 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:15.480260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=4 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:15.487164 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:15.487982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:15.488554 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:15.489466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:15.490346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:15.491024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.177142 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:16.177954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:16.189649 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:16.190337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:16.190782 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.191700 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:16.192156 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:16.192654 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.222685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:16.230453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f238feee-4724-4abf-ae3d-32cfff51ffae tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 0.757948 Sep 11 00:53:16.239518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 28/54] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:15 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:16.243921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:16.244538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:16.259797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:16.260645 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:16.261552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.262650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:16.264383 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:16.265030 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.562455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:16.582205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:16.584334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:16.587916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ed745df5-1485-4d30-b12d-416bd7ee9e54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.415951 Sep 11 00:53:16.592878 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:16.594136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 27/55] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:16 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:16.602104 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:16.602659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:16.617465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e04d6548-fff6-4bcd-99ad-d6c107b95b7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.377069 Sep 11 00:53:16.618048 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 29/56] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:16 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:16.618300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:16.618829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:16.619266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.619974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:16.620435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:16.620862 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:16.725992 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:16.738661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b31c7213-f772-40d6-a419-9feb0ced3805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1299 microversion: 2.1 time: 0.140379 Sep 11 00:53:16.751853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 28/57] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:16 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1299 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:17.262538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:17.262829 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:17.274115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.275853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.277208 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.279738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.280761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.281535 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.416092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-846a9fc8-1296-4984-9662-7643c79c7e07 req-17f71973-bd3a-45e7-8437-7178681df4e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9acff3d4-2dd1-4809-bc64-2dfc7829bfc6", "name": "network-vif-plugged", "status": "completed", "tag": "1fc55f70-880d-4007-99be-49ec52f1b61e"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:17.438490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-846a9fc8-1296-4984-9662-7643c79c7e07 req-17f71973-bd3a-45e7-8437-7178681df4e3 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.441122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-846a9fc8-1296-4984-9662-7643c79c7e07 req-17f71973-bd3a-45e7-8437-7178681df4e3 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.441768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-846a9fc8-1296-4984-9662-7643c79c7e07 req-17f71973-bd3a-45e7-8437-7178681df4e3 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.465079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-846a9fc8-1296-4984-9662-7643c79c7e07 req-17f71973-bd3a-45e7-8437-7178681df4e3 service nova] Creating event network-vif-plugged:1fc55f70-880d-4007-99be-49ec52f1b61e for instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 on hv-cinder-maste Sep 11 00:53:17.475360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-846a9fc8-1296-4984-9662-7643c79c7e07 req-17f71973-bd3a-45e7-8437-7178681df4e3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062974 Sep 11 00:53:17.484138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 29/58] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:53:17.499933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:17.508296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a74f3230-3e31-4a46-8030-f6b3f280b1c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 0.250718 Sep 11 00:53:17.523476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 30/59] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:17 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:17.627599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:17.644260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:17.645504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:17.652110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:17.672714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=4 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:17.683895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.685406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.686238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.686877 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.687532 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.688416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.689007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.689635 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.690511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:17.691134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.691807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:17.694705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:17.730028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:18.182578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:18.192101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:18.193546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:18.194669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-80d27553-369d-4f52-9b11-1c8aa003a8b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.578943 Sep 11 00:53:18.207579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 30/60] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:17 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:18.238970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:18.243327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:18.269887 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:18.273248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:18.273702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:18.279349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:18.286517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:18.287039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:18.342687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:18.351826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-191ebbe6-af87-4d48-bff6-4c16f3001ec4 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.714619 Sep 11 00:53:18.353720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 31/61] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:17 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 717 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:18.425387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:18.447788 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fa14fa98-405c-4dab-9425-dd11b16a21a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1299 microversion: 2.1 time: 0.216851 Sep 11 00:53:18.448418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 31/62] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:18 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1299 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:18.535685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:18.537814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:18.548223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:18.556866 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:18.557678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:18.559477 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:18.560404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:18.561047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:18.916745 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:18.931552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-aa40dcaf-926e-46ae-8a24-01e3507404ba tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1321 microversion: 2.1 time: 0.402843 Sep 11 00:53:18.932143 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 32/63] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:18 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1321 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:19.241371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:19.242069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:19.251249 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.252445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.253107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.253999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.254606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.255422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.378046 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:19.381419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:19.389869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.390592 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.391669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.392730 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.393662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.394416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.449686 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:19.461054 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:19.462751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:19.464066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fd7f1d21-7daa-479c-8242-b24124874b04 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.227345 Sep 11 00:53:19.465105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 32/64] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:19 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:19.472051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:19.472844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:19.489349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.490030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.490708 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.492840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.494385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.494848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.630678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:19.640170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-35112641-e779-44bd-8624-50d6bc1a8af6 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.267025 Sep 11 00:53:19.643167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 33/65] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:19 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:19.769030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:19.781115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7e6eb799-b8e9-4b62-a1e4-3d8923f56a97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.314172 Sep 11 00:53:19.781703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 33/66] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:19 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:19.966231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:19.968455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:19.978225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.978970 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.979657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:19.980602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:19.981417 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:19.982563 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.237408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:20.249512 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a37df664-f3b6-4715-9f37-fc46d90bf160 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1516 microversion: 2.1 time: 0.289053 Sep 11 00:53:20.255412 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 34/67] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:19 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1516 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:20.283098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:20.285696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:20.300532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.301040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.301482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.302198 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.302657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.303988 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.432306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-9b3d8e92-96d9-4838-acd6-93f01451b7da req-5d9e4de7-cdcf-4d9c-833e-8f6b5977e5d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d334d7b3-7a92-42bd-902a-39728c315ee9", "name": "network-vif-plugged", "status": "completed", "tag": "1a22d551-a8f3-4dd7-bcbb-3d16039c58ef"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:20.450190 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lazy-loading 'flavor' on Instance uuid 62f2702d-714f-47f4-87e5-25f0283ee9a5 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:53:20.464412 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-9b3d8e92-96d9-4838-acd6-93f01451b7da req-5d9e4de7-cdcf-4d9c-833e-8f6b5977e5d5 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.464955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-9b3d8e92-96d9-4838-acd6-93f01451b7da req-5d9e4de7-cdcf-4d9c-833e-8f6b5977e5d5 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.465402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-9b3d8e92-96d9-4838-acd6-93f01451b7da req-5d9e4de7-cdcf-4d9c-833e-8f6b5977e5d5 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.499098 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-9b3d8e92-96d9-4838-acd6-93f01451b7da req-5d9e4de7-cdcf-4d9c-833e-8f6b5977e5d5 service nova] Creating event network-vif-plugged:1a22d551-a8f3-4dd7-bcbb-3d16039c58ef for instance d334d7b3-7a92-42bd-902a-39728c315ee9 on hv-cinder-maste Sep 11 00:53:20.526094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-9b3d8e92-96d9-4838-acd6-93f01451b7da req-5d9e4de7-cdcf-4d9c-833e-8f6b5977e5d5 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097409 Sep 11 00:53:20.529395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 35/68] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:53:20.564929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:20.565942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:20.588893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.589435 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.591345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.593417 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.593908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.595847 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.759274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:20.771594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5ea85ecf-cf6d-4584-b7ea-00938b8c4de0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5/action" status: 200 len: 32 microversion: 2.1 time: 0.493775 Sep 11 00:53:20.776594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 34/69] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:53:20 2022] POST /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5/action => generated 32 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 00:53:20.785723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:20.788210 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:20.807469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.808624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.811517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:20.813999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:20.814619 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:20.815883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:21.038428 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:21.051242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:21.054867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:21.057182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f85bd620-8705-41f7-88c9-cedf97db1ebb tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.524773 Sep 11 00:53:21.058006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 36/70] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:20 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:21.064050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:21.065261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:21.072462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:21.074217 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:21.074887 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:21.076219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:21.076908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:21.078120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:21.112182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:21.125333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5ce634c4-68c2-4873-9f70-5b3fe0bfad59 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1314 microversion: 2.1 time: 0.347266 Sep 11 00:53:21.126562 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 35/71] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:20 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1314 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:21.132056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:21.132896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:21.140822 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:21.141423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:21.141986 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:21.142702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:21.143193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:21.143629 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:21.320046 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:21.336009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a02d18ff-eda0-4451-8fff-dec2a87f535e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.276371 Sep 11 00:53:21.338206 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 37/72] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:21 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:21.368667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-7c3037a2-0875-4f5c-9f5f-ff01fbfaa25f req-a89c7123-cbff-4487-a695-2b0cc94f4f0d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fb960ab4-4a34-4074-813f-cd971d916bcf", "tag": "41d6739a-7225-40b2-aedd-df0beff65d5f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:21.369136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:21.379023 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d3c04ce2-b4c3-4122-bab9-abc730871d25 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.251264 Sep 11 00:53:21.394080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7c3037a2-0875-4f5c-9f5f-ff01fbfaa25f req-a89c7123-cbff-4487-a695-2b0cc94f4f0d service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:21.394080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7c3037a2-0875-4f5c-9f5f-ff01fbfaa25f req-a89c7123-cbff-4487-a695-2b0cc94f4f0d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:21.394080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7c3037a2-0875-4f5c-9f5f-ff01fbfaa25f req-a89c7123-cbff-4487-a695-2b0cc94f4f0d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:21.394810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 36/73] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:21 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:21.414306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-7c3037a2-0875-4f5c-9f5f-ff01fbfaa25f req-a89c7123-cbff-4487-a695-2b0cc94f4f0d service nova] Creating event network-changed:41d6739a-7225-40b2-aedd-df0beff65d5f for instance fb960ab4-4a34-4074-813f-cd971d916bcf on hv-cinder-maste Sep 11 00:53:21.421055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-7c3037a2-0875-4f5c-9f5f-ff01fbfaa25f req-a89c7123-cbff-4487-a695-2b0cc94f4f0d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066584 Sep 11 00:53:21.422538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 38/74] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:22.087062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:22.087832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:22.104725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.106212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.108759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.111073 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.114387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.120065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.153459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:22.155094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:22.171211 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.171733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.172196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.173752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.174205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.177489 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.367639 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:22.376961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:22.378738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:22.379979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c7ce2b0f-dcdb-48a7-81f8-82e71a15e35f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.297920 Sep 11 00:53:22.380629 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 37/75] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:22 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:22.385430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:22.386177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:22.393248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.393945 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.394376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.395236 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.395817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.396365 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.416844 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:22.424871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7c421755-da04-4564-bf90-843600541d63 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1502 microversion: 2.1 time: 0.277279 Sep 11 00:53:22.425856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 39/76] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:22 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1502 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:22.438855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:22.440013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:22.448574 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.449493 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.450756 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.454353 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.455367 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.456082 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.631461 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:22.646413 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:22.659717 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5ae85e06-a2d4-4b50-bc15-6cfb559ae296 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.277830 Sep 11 00:53:22.664805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 38/77] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:22 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:22.677271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-64ccf49a-a4e6-4154-bef9-452cc041acf6 req-68632718-a02d-4afd-a9fd-d5f6b89668ab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "name": "network-vif-plugged", "status": "completed", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:22.698392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=14 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:22.705383 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:22.707266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-64ccf49a-a4e6-4154-bef9-452cc041acf6 req-68632718-a02d-4afd-a9fd-d5f6b89668ab service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:22.709212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-64ccf49a-a4e6-4154-bef9-452cc041acf6 req-68632718-a02d-4afd-a9fd-d5f6b89668ab service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:22.709905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-64ccf49a-a4e6-4154-bef9-452cc041acf6 req-68632718-a02d-4afd-a9fd-d5f6b89668ab service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:22.718034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ebd5ef69-06a2-4d45-b466-e7797a81350f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.290926 Sep 11 00:53:22.719204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 40/78] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:22 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:22.752893 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-64ccf49a-a4e6-4154-bef9-452cc041acf6 req-68632718-a02d-4afd-a9fd-d5f6b89668ab service nova] Creating event network-vif-plugged:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:53:22.759639 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-64ccf49a-a4e6-4154-bef9-452cc041acf6 req-68632718-a02d-4afd-a9fd-d5f6b89668ab service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093385 Sep 11 00:53:22.761476 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 39/79] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:23.406930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:23.407709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:23.413912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:23.414516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:23.415100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:23.415929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:23.416486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:23.417147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:23.584334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:23.594994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:23.595807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:23.596927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cc41da41-cc10-470c-bd61-b1616577f174 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.194228 Sep 11 00:53:23.597462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 41/80] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:23 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:23.693291 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:23.694176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:23.704653 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:23.705650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:23.706286 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:23.707453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:23.708063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:23.708798 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:23.748120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:23.748784 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:23.751015 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:23.759655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:23.760094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:23.764346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:23.765080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:23.765547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:23.765970 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:24.088409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:24.097668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ebbbe004-236e-4873-aa9b-56c19477bcb4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.409071 Sep 11 00:53:24.101922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 40/81] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:23 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:24.117867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4d89f42b-a488-406c-884f-1459a13a64f2"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:24.120377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:24.126583 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:24.127112 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:24.127709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:24.128718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:24.129184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:24.129617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:24.146359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:24.156180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-99c7f207-3764-4b84-a118-5f38b98e154c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.411823 Sep 11 00:53:24.156955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 42/82] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:23 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:24.623564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:24.624348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:24.630826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:24.631456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:24.631987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:24.632822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:24.635601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:24.636069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:24.900910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:24.913634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:24.916927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:24.918332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-29b315ee-3dec-4042-a4b4-f858d4058088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.299131 Sep 11 00:53:24.929706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 43/83] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:24 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:25.123001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:25.123001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:25.127968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:25.128621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:25.129204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:25.130068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:25.130673 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:25.131166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:25.303062 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:25.311563 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5c6184f1-0dff-4074-a797-cad72ebdf946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.194854 Sep 11 00:53:25.312297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 44/84] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:25 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:25.328623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:25.329241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:25.336388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:25.336388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:25.336631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:25.337333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:25.338074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:25.338510 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:25.498803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:25.507895 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fd9d1d1f-14a5-43ac-98ee-4da9f209bdd0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.194443 Sep 11 00:53:25.508626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 45/85] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:25 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:25.944740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:25.945472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:25.951864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:25.952598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:25.953251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:25.954138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:25.954655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:25.955332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:26.127899 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:26.136092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:26.137539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:26.139183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a6ffec34-3cb8-4837-9beb-a227dd1b363c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.198389 Sep 11 00:53:26.139742 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 46/86] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:25 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:53:26.339413 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:26.340267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:26.350565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:26.351134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:26.351572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:26.352322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:26.353069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:26.353496 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:26.536158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:26.549804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ab063a31-24a3-41d7-8927-c00d977d515f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.208944 Sep 11 00:53:26.550833 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 47/87] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:26 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:26.558173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:26.559358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:26.579456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:26.580049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:26.580692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:26.581469 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:26.581938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:26.582369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:26.744576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:26.757161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-47ef914b-a8f3-461e-a569-2163b3ec7d5c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.204359 Sep 11 00:53:26.758264 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 48/88] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:26 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:27.166838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:27.167481 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:27.176274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:27.176698 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:27.177131 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:27.177828 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:27.178399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:27.178859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:27.183138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:27.328022 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:27.336339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:27.338416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:27.339724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-be19f962-66fd-4970-8812-facb407d6750 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.178298 Sep 11 00:53:27.340409 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 49/89] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:27 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:53:27.573533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:27.574573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:27.593316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:27.594023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:27.595453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:27.599035 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:27.600359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:27.603055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:27.788139 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:27.798076 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9f094215-0cee-4fd2-afad-f47de2087fc4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.228352 Sep 11 00:53:27.816152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 50/90] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:27 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:27.837515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:27.838591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:27.845981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:27.846648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:27.847915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:27.865638 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:27.867569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:27.868189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:27.899388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-242f4658-1523-4827-b56c-7bd662d34065 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "POST /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.783747 Sep 11 00:53:27.901463 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 41/91] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:53:24 2022] POST /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9/os-volume_attachments => generated 194 bytes in 3790 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:53:28.146684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:28.158776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1d00c112-7962-4375-8450-bcdc2472b394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.342010 Sep 11 00:53:28.159856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 51/92] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:27 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:28.367556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:28.368569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:28.384641 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:28.385431 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:28.386074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:28.386978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:28.387718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:28.388441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:28.599173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:28.610651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:28.612420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:28.613743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6cdc909c-7c5f-4e7b-b551-822f606011ac tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1406 microversion: 2.20 time: 0.250938 Sep 11 00:53:28.614346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 42/93] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:28 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1406 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:28.834821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:28.838131 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:28.850107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:28.851489 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:28.852543 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:28.853511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:28.855631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:28.857920 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.074011 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:29.093068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4d8bf506-22d3-41e4-b39c-ea15bb905616 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.275292 Sep 11 00:53:29.094005 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 52/94] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:28 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:29.195855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:29.197290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:29.206929 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:29.207834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:29.208466 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.209883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:29.210594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:29.211095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.365539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:29.375928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3995b046-9ba1-4eba-b82e-cc47ffb494b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.194940 Sep 11 00:53:29.376513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 43/95] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:29 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:29.642149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:29.642764 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:29.649031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:29.649670 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:29.650340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.651070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:29.651816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:29.652249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.839826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:29.848680 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:29.850153 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:29.851345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ac85eb86-60ec-4c9e-8787-93f0b346e6f5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1597 microversion: 2.20 time: 0.213772 Sep 11 00:53:29.851910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 53/96] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:53:29 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1597 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:53:29.876531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:29.877196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:29.883734 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:29.884446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:29.889396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.890209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:29.890743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:29.891338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:29.974969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6f9e23c6-d888-4454-8eb3-3253de54c4ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.102689 Sep 11 00:53:29.975977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 44/97] 10.222.0.27 () {60 vars in 1316 bytes} [Sun Sep 11 00:53:29 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-interface => generated 284 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:53:30.117427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:30.118033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:30.126277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:30.126772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:30.127679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:30.129028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:30.129501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:30.130037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:30.403187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:30.405111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:30.412325 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:30.425764 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-21f60ce5-b972-49e3-8546-a781144f6363 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.312032 Sep 11 00:53:30.428490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 54/98] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:30 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:30.429756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:30.430287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:30.430725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:30.431617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:30.432076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:30.432503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:31.088997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:31.099496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ce98dd63-ea72-4ff9-b78a-f09b39c47773 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.702551 Sep 11 00:53:31.100312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 45/99] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:30 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:31.453502 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:31.454663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:31.460740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:31.461303 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:31.461975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:31.462858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:31.463480 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:31.463926 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:31.705657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:31.725690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d5eda4f6-28f5-4171-84c5-9aa4d25ff954 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.275750 Sep 11 00:53:31.728685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 55/100] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:31 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:31.830452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-f4e2884e-89e4-495b-9784-fd0f5a13cce2 req-c85d4ebc-81c4-4a9a-bb64-bfaeed3bcbf7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:31.841874 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f4e2884e-89e4-495b-9784-fd0f5a13cce2 req-c85d4ebc-81c4-4a9a-bb64-bfaeed3bcbf7 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:31.842462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f4e2884e-89e4-495b-9784-fd0f5a13cce2 req-c85d4ebc-81c4-4a9a-bb64-bfaeed3bcbf7 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:31.843061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f4e2884e-89e4-495b-9784-fd0f5a13cce2 req-c85d4ebc-81c4-4a9a-bb64-bfaeed3bcbf7 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:31.858146 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-f4e2884e-89e4-495b-9784-fd0f5a13cce2 req-c85d4ebc-81c4-4a9a-bb64-bfaeed3bcbf7 service nova] Creating event network-changed:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:53:31.863321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-f4e2884e-89e4-495b-9784-fd0f5a13cce2 req-c85d4ebc-81c4-4a9a-bb64-bfaeed3bcbf7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039639 Sep 11 00:53:31.866627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 46/101] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:32.128807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:32.129681 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:32.136212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:32.136910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:32.137554 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:32.138774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:32.139395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:32.140709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:32.326954 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:32.337475 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f60c74a4-61f7-49ef-8ec6-7620bb23514e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.212428 Sep 11 00:53:32.338438 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 56/102] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:32 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:32.469632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-a70e29d2-9024-402c-90fa-975b92c95596 req-4df8f12e-0e88-46ff-b37d-a0545c19cae1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fb960ab4-4a34-4074-813f-cd971d916bcf", "name": "network-vif-plugged", "status": "completed", "tag": "41d6739a-7225-40b2-aedd-df0beff65d5f"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:32.506372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a70e29d2-9024-402c-90fa-975b92c95596 req-4df8f12e-0e88-46ff-b37d-a0545c19cae1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:32.507615 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a70e29d2-9024-402c-90fa-975b92c95596 req-4df8f12e-0e88-46ff-b37d-a0545c19cae1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:32.510645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a70e29d2-9024-402c-90fa-975b92c95596 req-4df8f12e-0e88-46ff-b37d-a0545c19cae1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:32.543898 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-a70e29d2-9024-402c-90fa-975b92c95596 req-4df8f12e-0e88-46ff-b37d-a0545c19cae1 service nova] Creating event network-vif-plugged:41d6739a-7225-40b2-aedd-df0beff65d5f for instance fb960ab4-4a34-4074-813f-cd971d916bcf on hv-cinder-maste Sep 11 00:53:32.560938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-a70e29d2-9024-402c-90fa-975b92c95596 req-4df8f12e-0e88-46ff-b37d-a0545c19cae1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096147 Sep 11 00:53:32.562871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 47/103] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:32.750425 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:32.751854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:32.760789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:32.763791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:32.765069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:32.767050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:32.767684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:32.769268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:33.031688 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:33.047099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-64d03f74-526e-4850-9abf-63f49c33aee8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.300989 Sep 11 00:53:33.048974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 57/104] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:32 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:33.362946 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:33.363877 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:33.370662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:33.371472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:33.372136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:33.372957 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:33.374290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:33.374952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:33.614331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:33.630595 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-94f8bfe9-76e5-4bfa-a047-91325edaf875 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.272374 Sep 11 00:53:33.631661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 48/105] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:33 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:34.081980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:34.081980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:34.083097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:34.083896 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:34.085033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:34.085885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:34.086486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:34.087081 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:34.238089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:34.246228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d7755938-e57c-4b83-aafb-2a3b4174cd85 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.174490 Sep 11 00:53:34.246812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 58/106] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:34 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:34.654716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:34.655445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:34.662690 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:34.663468 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:34.664054 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:34.665019 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:34.665625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:34.666265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:34.841916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:34.854567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cdca9e62-4011-42d4-bb23-e98de140adda tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.203476 Sep 11 00:53:34.855329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 49/107] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:34 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:35.270479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:35.271914 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:35.279643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:35.280427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:35.281043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:35.281946 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:35.282679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:35.283028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:35.449518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:35.458306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b8598790-30cb-4947-b64c-d296702519d5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.191648 Sep 11 00:53:35.458902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 59/108] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:35 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:35.877970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:35.878639 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:35.885071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:35.885555 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:35.885991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:35.886679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:35.887706 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:35.888371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:36.055309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:36.064744 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e6ff6d10-413a-422c-920c-180e163ee754 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.190522 Sep 11 00:53:36.065409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 50/109] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:35 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:36.484236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:36.484856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:36.491957 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:36.492455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:36.492887 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:36.494038 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:36.494515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:36.494973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:36.647109 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:36.656617 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-92c159ea-89ad-4438-b538-d5471a2d46e6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.177769 Sep 11 00:53:36.657388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 60/110] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:36 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:37.093297 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:37.094062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:37.102310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:37.110885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:37.111259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:37.112166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:37.112433 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:37.112857 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:37.289076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:37.297914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5d78159c-c2d5-49e6-96d4-b0b3ffb52050 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.209362 Sep 11 00:53:37.298477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 51/111] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:37 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:37.680421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:37.681155 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:37.687573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:37.688060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:37.688572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:37.689275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:37.689728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:37.690149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:37.878379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:37.888034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-af14d689-33ea-4320-9651-ea207dbcd27b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.210941 Sep 11 00:53:37.888930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 61/112] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:37 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:38.321273 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:38.322011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:38.324472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:38.330482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:38.331003 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:38.331441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:38.332170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:38.332625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:38.333046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:38.348499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:INSERT=2,nova_cell1:SELECT=96,nova_cell1:UPDATE=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:38.471344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:38.478833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2c678c55-dfc1-4c66-87c5-267ea3af414c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1367 microversion: 2.1 time: 0.161360 Sep 11 00:53:38.479395 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 52/113] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:38 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1367 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:38.913163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:38.914722 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:38.920799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:38.921460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:38.922057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:38.922968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:38.923565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:38.924139 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:39.129215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:39.136853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-38f1b9b4-e5d0-4896-8254-a0a27beb00e9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.228212 Sep 11 00:53:39.137522 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 62/114] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:38 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:39.504034 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:39.505303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:39.513697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:39.514525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:39.515733 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:39.516627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:39.517079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:39.517669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:39.657587 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:39.665731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4de2841a-7c27-4091-921f-ea48a1286622 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1557 microversion: 2.1 time: 0.166684 Sep 11 00:53:39.666292 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 53/115] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:39 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1557 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:39.689822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:39.690318 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:39.696123 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:39.696606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:39.697040 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:39.697746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:39.698201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:39.698623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:39.817596 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-357a7c04-5407-42e4-a79f-bd18196c5805 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.131880 Sep 11 00:53:39.818270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 63/116] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 00:53:39 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-interface => generated 284 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:40.159405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:40.160420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:40.170707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:40.171320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:40.172022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:40.172784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:40.173308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:40.173732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:40.598560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:40.608270 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c9f23104-3d56-46a5-a6ee-91e90b804103 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.452740 Sep 11 00:53:40.608940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 54/117] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:40 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:41.046849 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-ff14acb8-4e6e-4725-9be6-440189e586d9 req-72fc7ed4-5ef0-4d46-a467-50f2d33fe2e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fb960ab4-4a34-4074-813f-cd971d916bcf", "tag": "41d6739a-7225-40b2-aedd-df0beff65d5f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:41.056418 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ff14acb8-4e6e-4725-9be6-440189e586d9 req-72fc7ed4-5ef0-4d46-a467-50f2d33fe2e1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:41.056892 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ff14acb8-4e6e-4725-9be6-440189e586d9 req-72fc7ed4-5ef0-4d46-a467-50f2d33fe2e1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:41.057317 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ff14acb8-4e6e-4725-9be6-440189e586d9 req-72fc7ed4-5ef0-4d46-a467-50f2d33fe2e1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:41.068855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-ff14acb8-4e6e-4725-9be6-440189e586d9 req-72fc7ed4-5ef0-4d46-a467-50f2d33fe2e1 service nova] Creating event network-changed:41d6739a-7225-40b2-aedd-df0beff65d5f for instance fb960ab4-4a34-4074-813f-cd971d916bcf on hv-cinder-maste Sep 11 00:53:41.074187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-ff14acb8-4e6e-4725-9be6-440189e586d9 req-72fc7ed4-5ef0-4d46-a467-50f2d33fe2e1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031456 Sep 11 00:53:41.075509 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 64/118] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:41.631844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:41.632449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:41.639578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:41.640166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:41.640694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:41.641457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:41.641981 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:41.642470 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:41.771673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:41.779606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9c89ede1-9d29-47f6-8d43-85143bae5a45 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.151668 Sep 11 00:53:41.780170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 55/119] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:41 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:42.804228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:42.805338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:42.813452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:42.813943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:42.814389 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:42.815289 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:42.815758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:42.816188 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:42.968364 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:42.977021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5248bc31-0773-41b8-a65e-5487b5d8c710 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.177038 Sep 11 00:53:42.977595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 65/120] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:42 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:44.004047 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:44.004666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:44.011148 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:44.011639 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:44.012589 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:44.013277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:44.013732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:44.014157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:44.156611 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:44.164541 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1396500c-427c-400e-b0fc-8ed14f9af07d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.164279 Sep 11 00:53:44.165097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 56/121] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:43 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:45.187659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:45.188274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:45.194193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:45.194671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:45.195133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:45.195810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:45.196260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:45.196679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:45.326995 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:45.330454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:45.335493 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0c34e8fb-cc05-47e2-89c2-ce6f150a863f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.151927 Sep 11 00:53:45.336057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 66/122] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:45 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:46.357765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:46.358384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:46.364835 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:46.365321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:46.365760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:46.366450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:46.366942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:46.367455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:46.507301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:46.515205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-536a8f71-0820-41d5-93b7-a0ab4ac33a38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.161708 Sep 11 00:53:46.515766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 57/123] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:46 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:47.546703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:47.551210 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:47.553520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=78 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:47.568291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:47.568789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:47.569226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:47.581316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:47.581849 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:47.582260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:47.773173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:47.781272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-68032b21-4fd4-4753-93b2-cf87c4abb339 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.245353 Sep 11 00:53:47.781843 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 67/124] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:47 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:48.677089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:48.677575 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:48.683914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:48.684400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:48.684836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:48.685528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:48.685980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:48.686399 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:48.712474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:53:48.805333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:48.806175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:48.807032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-53c253e2-2c29-4132-8b1a-49e4d035117f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 204 len: 0 microversion: 2.1 time: 0.133398 Sep 11 00:53:48.807853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 58/125] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:53:48 2022] DELETE /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:53:48.814234 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:48.815224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:48.816021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:48.816997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:48.817423 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:48.817759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:48.818362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:48.819052 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:48.823616 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:48.824154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:48.825135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:48.825838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:48.826295 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:48.826722 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:49.034701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:49.045570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-642e01b3-ff70-46be-91e2-eac581d683ea tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.232898 Sep 11 00:53:49.046876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 59/126] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:48 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:49.055801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:49.067994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6bac17b7-4926-408b-8add-a71139b549fa tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.266947 Sep 11 00:53:49.068672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 68/127] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:48 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:50.069694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:50.070241 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:50.082272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:50.082272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:50.082272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:50.082272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:50.082762 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:50.082762 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:50.092731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:50.093429 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:50.100407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:50.101249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:50.101853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:50.102701 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:50.103488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:50.104213 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:50.295692 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:50.303924 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6c39f945-d9ec-4ab1-895a-691da1be9d9d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.238277 Sep 11 00:53:50.304429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 60/128] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:50 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:50.313908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:50.321971 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fd840b35-a78f-447e-ba9c-20f4e065d597 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.233783 Sep 11 00:53:50.322532 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 69/129] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:50 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:51.330306 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:51.331586 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:51.338400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:51.341039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:51.341039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:51.341039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:51.341557 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:51.347168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:51.347788 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:51.348108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:51.354226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:51.354830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:51.355356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:51.356233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:51.356862 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:51.357337 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:51.552118 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:51.556270 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:51.560961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8cfa20d5-b59a-4fc1-8edb-6cb1a54a2b42 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.217797 Sep 11 00:53:51.561822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 70/130] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:51 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:51.565362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b3b686eb-c8ae-4ec9-afa5-883f8da246d5 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.241415 Sep 11 00:53:51.565916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 61/131] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:51 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:52.588324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:52.588958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:52.592233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:52.592649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:52.599345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:52.599893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:52.600162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:52.600322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:52.600650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:52.601088 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:52.601784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:52.602309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:52.602598 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:52.602768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:52.603159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:52.604390 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:52.811614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:52.814817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:52.820539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bf62fd0d-ffd5-4090-899f-ec9bcbcaa5a1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.240357 Sep 11 00:53:52.821228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 71/132] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:52 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:52.823418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0c551a4d-d4f0-4b33-b758-519a4cd184fa tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.238908 Sep 11 00:53:52.824611 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 62/133] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:52 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:53.845378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:53.845991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:53.846741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:53.847372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:53.852104 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:53.852604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:53.853034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:53.853718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:53.853812 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:53.854165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:53.854325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:53.854584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:53.854813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:53.855671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:53.856325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:53.856759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:54.053862 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:54.056677 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:54.065645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4cd47cae-499f-4ca9-b0df-7e0f14fa7724 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.222858 Sep 11 00:53:54.066287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 63/134] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:53 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:54.067692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ac24e150-3a2d-4d12-91a3-f97b0c1f9285 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.226183 Sep 11 00:53:54.073207 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 72/135] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:53 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:55.091074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:55.091708 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:55.095903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:55.096761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:55.097861 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:55.098358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:55.104007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:55.104007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:55.104007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:55.104007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:55.104497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:55.104497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:55.104497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:55.105772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:55.106443 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:55.106985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:55.297446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:55.301147 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:55.305267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9e18760e-2e24-4119-b7bc-af06327c6a32 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.218184 Sep 11 00:53:55.305966 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 73/136] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:55 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:55.312761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2ea1d4e5-8753-4886-8196-17ac28db0a1b tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.221480 Sep 11 00:53:55.313389 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 64/137] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:55 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:56.329274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:56.329989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:56.338839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:56.339644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:56.340401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:56.340676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:56.340824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:56.341825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:56.342380 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:56.342861 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:56.353188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:56.353810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:56.354388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:56.355246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:56.355968 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:56.356504 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:56.546629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:56.554375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a828fab0-152d-41eb-8165-4489c6a79117 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.230386 Sep 11 00:53:56.555272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 74/138] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:56 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:56.575582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:56.585310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f9250936-a976-4678-b708-4483b263df0f tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1554 microversion: 2.1 time: 0.252537 Sep 11 00:53:56.585991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 65/139] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:56 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1554 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:57.485611 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-3343d413-b994-4996-85b1-2a96db7890ab req-d2003b86-3432-4726-8ad3-d04bd1fed962 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d334d7b3-7a92-42bd-902a-39728c315ee9", "name": "network-vif-deleted", "tag": "1a22d551-a8f3-4dd7-bcbb-3d16039c58ef"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:53:57.498015 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3343d413-b994-4996-85b1-2a96db7890ab req-d2003b86-3432-4726-8ad3-d04bd1fed962 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:57.498725 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3343d413-b994-4996-85b1-2a96db7890ab req-d2003b86-3432-4726-8ad3-d04bd1fed962 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:57.499326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3343d413-b994-4996-85b1-2a96db7890ab req-d2003b86-3432-4726-8ad3-d04bd1fed962 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:57.517902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-3343d413-b994-4996-85b1-2a96db7890ab req-d2003b86-3432-4726-8ad3-d04bd1fed962 service nova] Creating event network-vif-deleted:1a22d551-a8f3-4dd7-bcbb-3d16039c58ef for instance d334d7b3-7a92-42bd-902a-39728c315ee9 on hv-cinder-maste Sep 11 00:53:57.524218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-3343d413-b994-4996-85b1-2a96db7890ab req-d2003b86-3432-4726-8ad3-d04bd1fed962 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043946 Sep 11 00:53:57.525037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 75/140] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:53:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:53:57.585015 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:57.585869 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:57.594481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:57.595001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:57.595433 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:57.596123 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:57.596570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:57.596994 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:57.611431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:57.612343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:57.630303 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:57.630816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:57.631272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:57.631974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:57.632424 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:57.632836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:57.769542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:57.782369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-33dd4307-2951-4fa8-8278-6645ff9e6ed2 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1345 microversion: 2.1 time: 0.177601 Sep 11 00:53:57.794938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 76/141] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:57 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1345 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:57.853596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:57.863259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c047827d-e696-4a5c-a091-e3244b691717 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.285833 Sep 11 00:53:57.864101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 66/142] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:57 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:58.811137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:58.812904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:58.820749 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:58.821242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:58.821740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:58.824408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:58.825479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:58.825959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:58.862802 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:58.863575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:58.864125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:58.864481 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lazy-loading 'fault' on Instance uuid d334d7b3-7a92-42bd-902a-39728c315ee9 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:53:58.889174 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:58.889913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:58.900563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:58.901825 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:58.903527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:58.904708 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:58.905709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:58.906209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:58.923745 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:58.942544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b1fc2c958f045538fb15d7d1d4a408a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '832fdbe87ca847f094c6cc8a1fc43a59', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:58.958763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d561a59a-8ba2-4059-9052-88de3e929c09 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 200 len: 1350 microversion: 2.1 time: 0.151651 Sep 11 00:53:58.963791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 77/143] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:58 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 1350 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:53:59.104137 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:53:59.114363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3c308fa2-5aa0-44c4-9d2e-07bd3381f4c8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.231295 Sep 11 00:53:59.115049 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 67/144] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:58 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:53:59.989719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:53:59.990359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:53:59.997117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:59.997681 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:59.998085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:53:59.998818 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:53:59.999282 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:53:59.999704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:00.014399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] HTTP exception thrown: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Request method failure captured: Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 HTTP/1.1 Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFkHikq-c3sTN1cTLd7v1amMIVNUBTHXZSFbm_sb-MRv6ry1x7uWoo8k3OhxkRURixPNuo9WCrCtCc0kDkkK_mrAuEKMiDqcWiwJNurnRBmzPZ5pYsMS6dBUPw_ADwwaDNY1DTRJ70snUeLUQ3QiSsE1JtaL-4z8LHHj4S8f2Ndqhw Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:54:00.019595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 8b1fc2c958f045538fb15d7d1d4a408a Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.020812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'd334d7b3-7a92-42bd-902a-39728c315ee9'} Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:00.021504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.023074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.024037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Returning 404 to user: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:00.024037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-096e6749-6bcd-466c-9716-f581350dd621 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 404 len: 111 microversion: 2.1 time: 0.035414 Sep 11 00:54:00.024037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 78/145] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:53:59 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:54:00.138450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:00.139166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:00.144084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:00.145301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:00.145788 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:00.146222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:00.146352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:00.146930 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:00.147381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:00.147801 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:00.153570 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:00.154068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:00.154621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:00.155532 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:00.156149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:00.156920 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:00.173497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] HTTP exception thrown: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Request method failure captured: Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9/os-volume_attachments HTTP/1.1 Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFkHikq-c3sTN1cTLd7v1amMIVNUBTHXZSFbm_sb-MRv6ry1x7uWoo8k3OhxkRURixPNuo9WCrCtCc0kDkkK_mrAuEKMiDqcWiwJNurnRBmzPZ5pYsMS6dBUPw_ADwwaDNY1DTRJ70snUeLUQ3QiSsE1JtaL-4z8LHHj4S8f2Ndqhw Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:54:00.175841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 8b1fc2c958f045538fb15d7d1d4a408a Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.177020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'server_id': 'd334d7b3-7a92-42bd-902a-39728c315ee9'} Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:00.177667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:00.178768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:00.181212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Returning 404 to user: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:00.182251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1a968e37-f888-4bc8-af3f-5d71351cd2e0 tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.042303 Sep 11 00:54:00.187279 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 79/146] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:54:00 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9/os-volume_attachments => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:54:00.316623 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:00.328893 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-29ed7d78-1bbe-4bd5-a0fc-ad998f224af2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.195230 Sep 11 00:54:00.330172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 68/147] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:00 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:01.352432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:01.353055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:01.359781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:01.360277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:01.360714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:01.361422 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:01.362055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:01.362487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:01.437232 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:01.439305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:01.447833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:01.449108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:01.450104 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:01.451080 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:01.451803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:01.452432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:01.491703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] HTTP exception thrown: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Request method failure captured: Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 HTTP/1.1 Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFkHikq-c3sTN1cTLd7v1amMIVNUBTHXZSFbm_sb-MRv6ry1x7uWoo8k3OhxkRURixPNuo9WCrCtCc0kDkkK_mrAuEKMiDqcWiwJNurnRBmzPZ5pYsMS6dBUPw_ADwwaDNY1DTRJ70snUeLUQ3QiSsE1JtaL-4z8LHHj4S8f2Ndqhw Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:54:01.498596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 8b1fc2c958f045538fb15d7d1d4a408a Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.500002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'd334d7b3-7a92-42bd-902a-39728c315ee9'} Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:01.500707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.501749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.507005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Returning 404 to user: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:01.508021 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c1489d8c-43c7-4bc2-9e03-e7429fb008fd tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 404 len: 111 microversion: 2.1 time: 0.075974 Sep 11 00:54:01.509317 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 69/148] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:54:01 2022] DELETE /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:54:01.518274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:01.518921 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] [instance: d334d7b3-7a92-42bd-902a-39728c315ee9] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:01.531342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:01.531848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:01.532370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:01.533060 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:01.533515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:01.533953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:01.548931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:01.551237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] HTTP exception thrown: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Request method failure captured: Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 HTTP/1.1 Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFkHikq-c3sTN1cTLd7v1amMIVNUBTHXZSFbm_sb-MRv6ry1x7uWoo8k3OhxkRURixPNuo9WCrCtCc0kDkkK_mrAuEKMiDqcWiwJNurnRBmzPZ5pYsMS6dBUPw_ADwwaDNY1DTRJ70snUeLUQ3QiSsE1JtaL-4z8LHHj4S8f2Ndqhw Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:54:01.566048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_832fdbe87ca847f094c6cc8a1fc43a59"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 832fdbe87ca847f094c6cc8a1fc43a59 Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-DeleteServersTestJSON-1130876020 Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 8b1fc2c958f045538fb15d7d1d4a408a Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-DeleteServersTestJSON-1130876020-project Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.567170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'd334d7b3-7a92-42bd-902a-39728c315ee9'} Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:01.567834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. Sep 11 00:54:01.568870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:01.569886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] Returning 404 to user: Instance d334d7b3-7a92-42bd-902a-39728c315ee9 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:01.569886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d5c01719-524b-481a-8893-43c863916c7d tempest-DeleteServersTestJSON-1130876020 tempest-DeleteServersTestJSON-1130876020-project] 10.222.0.27 "GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9" status: 404 len: 111 microversion: 2.1 time: 0.040969 Sep 11 00:54:01.569886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 70/149] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:01 2022] GET /compute/v2.1/servers/d334d7b3-7a92-42bd-902a-39728c315ee9 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 00:54:01.570090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-363e178d-1a3b-4cd5-be32-cd198622fd7b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.220972 Sep 11 00:54:01.571167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 80/150] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:01 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:54:02.594229 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:02.595579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:02.608042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:02.609038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:02.609657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:02.610579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:02.611078 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:02.611671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:03.062278 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:03.072244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8a9cd80b-3038-4c2e-a6c3-4c699bef76a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.482128 Sep 11 00:54:03.078924 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 71/151] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:02 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:04.097201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:04.098100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:04.111819 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:04.112523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:04.113120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:04.113982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:04.114590 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:04.115392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:04.331498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:04.340101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-96172cb1-4f12-4ea0-b56f-7dc3ca39946d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.247137 Sep 11 00:54:04.340731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 81/152] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:04 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:05.369063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:05.370841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:05.385248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:05.385749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:05.387769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:05.388702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:05.389355 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:05.391355 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:05.630248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:05.648269 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8b1f037b-6ae9-41d9-b787-e2e0de2182f7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.288555 Sep 11 00:54:05.649511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 72/153] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:05 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:06.673349 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:06.674526 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:06.680571 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:06.681218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:06.681647 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:06.682338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:06.682968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:06.683683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:06.827675 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:06.836091 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c8fc8c22-dd9b-4020-acc8-c97e9c207748 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.167151 Sep 11 00:54:06.836697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 82/154] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:06 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:07.086668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-edcc95a3-95b0-4300-916c-dd642582faf2 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Action: 'create', calling method: .version_select at 0x7fc01ad3ef70>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-152169699", "metadata": {"Type": "work"}}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-edcc95a3-95b0-4300-916c-dd642582faf2 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Request method failure captured: Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 121 Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTGufZF0JlHHF5seF8zA_ZwtovzPhiKAJ3RzumU0fSsjkEhGTWFVZoVxy0vnxepP-8rC7qEYPNiRm13SixCvYQy87_7-BK9wmzeqEgjaP1f_81UFbTPzciKOfoZNmtjkqBiCuW-eL9bIsbhEO5-bKksTdKT3F5Mn5vueCuJ8Pe6AspE Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:54:07.097618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: f5c7de0bf5b4441cb887eb8822a1bb71 Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-152169699", "metadata": {"Type": "work"}}} Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 00:54:07.099160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-152169699', 'metadata': {'Type': 'work'}}}} Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise error Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi '#$%' Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:07.100789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 11 00:54:07.102204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 11 00:54:07.103323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 11 00:54:07.103323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.103323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-edcc95a3-95b0-4300-916c-dd642582faf2 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:07.103323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-edcc95a3-95b0-4300-916c-dd642582faf2 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.134128 Sep 11 00:54:07.103323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 73/155] 10.222.0.27 () {60 vars in 1156 bytes} [Sun Sep 11 00:54:06 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 135 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 11 00:54:07.109585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5c7a7394-0189-4692-9f72-1f3105c319b5 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Action: 'create', calling method: .version_select at 0x7fc01ade3280>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1482309798", "metadata": {"Type": "work"}}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5c7a7394-0189-4692-9f72-1f3105c319b5 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Request method failure captured: Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 120 Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTGufZF0JlHHF5seF8zA_ZwtovzPhiKAJ3RzumU0fSsjkEhGTWFVZoVxy0vnxepP-8rC7qEYPNiRm13SixCvYQy87_7-BK9wmzeqEgjaP1f_81UFbTPzciKOfoZNmtjkqBiCuW-eL9bIsbhEO5-bKksTdKT3F5Mn5vueCuJ8Pe6AspE Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:54:07.114996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: f5c7de0bf5b4441cb887eb8822a1bb71 Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1482309798", "metadata": {"Type": "work"}}} Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: Sep 11 00:54:07.116358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1482309798', 'metadata': {'Type': 'work'}}}} Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise error Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi '0' Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:07.117080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 11 00:54:07.118548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 11 00:54:07.119625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 11 00:54:07.119625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.119625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5c7a7394-0189-4692-9f72-1f3105c319b5 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:07.119625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5c7a7394-0189-4692-9f72-1f3105c319b5 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010908 Sep 11 00:54:07.119625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 83/156] 10.222.0.27 () {60 vars in 1156 bytes} [Sun Sep 11 00:54:07 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 13 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 11 00:54:07.126723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7bcf04e4-fc2c-49c5-848e-3448898064ca tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Action: 'create', calling method: .version_select at 0x7fc01addc790>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-745947340", "metadata": {"Type": "work"}}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7bcf04e4-fc2c-49c5-848e-3448898064ca tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Request method failure captured: Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 118 Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTGufZF0JlHHF5seF8zA_ZwtovzPhiKAJ3RzumU0fSsjkEhGTWFVZoVxy0vnxepP-8rC7qEYPNiRm13SixCvYQy87_7-BK9wmzeqEgjaP1f_81UFbTPzciKOfoZNmtjkqBiCuW-eL9bIsbhEO5-bKksTdKT3F5Mn5vueCuJ8Pe6AspE Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:54:07.130372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: f5c7de0bf5b4441cb887eb8822a1bb71 Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-745947340", "metadata": {"Type": "work"}}} Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 00:54:07.131789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-745947340', 'metadata': {'Type': 'work'}}}} Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise error Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi '' Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:07.132870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 11 00:54:07.134485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 11 00:54:07.136032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 11 00:54:07.136032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.136032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7bcf04e4-fc2c-49c5-848e-3448898064ca tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:07.136032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7bcf04e4-fc2c-49c5-848e-3448898064ca tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009046 Sep 11 00:54:07.136032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 74/157] 10.222.0.27 () {60 vars in 1156 bytes} [Sun Sep 11 00:54:07 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 11 00:54:07.141094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-631493aa-da1d-415b-8bd9-6aac349d0913 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Calling method '.version_select at 0x7fc01adfca60>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:07.171659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-631493aa-da1d-415b-8bd9-6aac349d0913 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] HTTP exception thrown: Volume tempest-invalid-2042937047 could not be found. Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-631493aa-da1d-415b-8bd9-6aac349d0913 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Request method failure captured: Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-2042937047 HTTP/1.1 Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTGufZF0JlHHF5seF8zA_ZwtovzPhiKAJ3RzumU0fSsjkEhGTWFVZoVxy0vnxepP-8rC7qEYPNiRm13SixCvYQy87_7-BK9wmzeqEgjaP1f_81UFbTPzciKOfoZNmtjkqBiCuW-eL9bIsbhEO5-bKksTdKT3F5Mn5vueCuJ8Pe6AspE Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:54:07.174901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: f5c7de0bf5b4441cb887eb8822a1bb71 Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Volume tempest-invalid-2042937047 could not be found. Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'tempest-invalid-2042937047'} Sep 11 00:54:07.176297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-2042937047 could not be found. (HTTP 404) (Request-ID: req-b224637f-e309-44ea-88e1-6b6a79c77047) Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.177249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 11 00:54:07.178556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-2042937047 could not be found. Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:07.179789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-2042937047 could not be found. Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-2042937047 could not be found. Sep 11 00:54:07.181394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.183932 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-631493aa-da1d-415b-8bd9-6aac349d0913 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Returning 404 to user: Volume tempest-invalid-2042937047 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:07.185191 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-631493aa-da1d-415b-8bd9-6aac349d0913 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/tempest-invalid-2042937047" status: 404 len: 99 microversion: 2.1 time: 0.048163 Sep 11 00:54:07.186453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 84/158] 10.222.0.27 () {58 vars in 1218 bytes} [Sun Sep 11 00:54:07 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-2042937047 => generated 99 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 11 00:54:07.196081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-82bb03ef-9b81-4273-a7d6-c4f525012b39 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003638 Sep 11 00:54:07.196736 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 75/159] 10.222.0.27 () {58 vars in 1140 bytes} [Sun Sep 11 00:54:07 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 11 00:54:07.206043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bcd2ef90-c0ef-49ed-bd89-9165eb9012d8 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004437 Sep 11 00:54:07.207043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 85/160] 10.222.0.27 () {58 vars in 1137 bytes} [Sun Sep 11 00:54:07 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 11 00:54:07.215391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8c8c40d2-49fa-4606-8219-e6cf6afa2246 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Calling method '.version_select at 0x7fc01af10550>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:07.243964 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-8c8c40d2-49fa-4606-8219-e6cf6afa2246 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] HTTP exception thrown: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8c8c40d2-49fa-4606-8219-e6cf6afa2246 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Request method failure captured: Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/os-volumes/fc0392a8-599c-4a0b-8c9d-eeed12708063 HTTP/1.1 Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTGufZF0JlHHF5seF8zA_ZwtovzPhiKAJ3RzumU0fSsjkEhGTWFVZoVxy0vnxepP-8rC7qEYPNiRm13SixCvYQy87_7-BK9wmzeqEgjaP1f_81UFbTPzciKOfoZNmtjkqBiCuW-eL9bIsbhEO5-bKksTdKT3F5Mn5vueCuJ8Pe6AspE Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:54:07.248657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: f5c7de0bf5b4441cb887eb8822a1bb71 Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'fc0392a8-599c-4a0b-8c9d-eeed12708063'} Sep 11 00:54:07.250079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. (HTTP 404) (Request-ID: req-4ea0b4b0-c3b4-4dd6-9e4f-452681d09cda) Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.250977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 11 00:54:07.252046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 11 00:54:07.253055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8c8c40d2-49fa-4606-8219-e6cf6afa2246 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Returning 404 to user: Volume fc0392a8-599c-4a0b-8c9d-eeed12708063 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8c8c40d2-49fa-4606-8219-e6cf6afa2246 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/fc0392a8-599c-4a0b-8c9d-eeed12708063" status: 404 len: 109 microversion: 2.1 time: 0.038846 Sep 11 00:54:07.254069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 76/161] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:54:07 2022] DELETE /compute/v2.1/os-volumes/fc0392a8-599c-4a0b-8c9d-eeed12708063 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:54:07.261433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5a2ea743-24af-451e-b58d-81e3b727a3c8 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Calling method '.version_select at 0x7fc01adfa160>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:07.294203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-5a2ea743-24af-451e-b58d-81e3b727a3c8 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] HTTP exception thrown: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5a2ea743-24af-451e-b58d-81e3b727a3c8 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Request method failure captured: Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/os-volumes/3cf172dc-de43-4a1c-bcb8-28f272dec8cf HTTP/1.1 Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTGufZF0JlHHF5seF8zA_ZwtovzPhiKAJ3RzumU0fSsjkEhGTWFVZoVxy0vnxepP-8rC7qEYPNiRm13SixCvYQy87_7-BK9wmzeqEgjaP1f_81UFbTPzciKOfoZNmtjkqBiCuW-eL9bIsbhEO5-bKksTdKT3F5Mn5vueCuJ8Pe6AspE Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:54:07.296180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_31655ad33439478a9a9bf0582cc908a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 31655ad33439478a9a9bf0582cc908a1 Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesNegativeTest-1800043371 Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: f5c7de0bf5b4441cb887eb8822a1bb71 Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesNegativeTest-1800043371-project Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '3cf172dc-de43-4a1c-bcb8-28f272dec8cf'} Sep 11 00:54:07.297403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. (HTTP 404) (Request-ID: req-756cea69-5fd0-4767-8b63-4e06a77b0be4) Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.298183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:54:07.299765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 11 00:54:07.301067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5a2ea743-24af-451e-b58d-81e3b727a3c8 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] Returning 404 to user: Volume 3cf172dc-de43-4a1c-bcb8-28f272dec8cf could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5a2ea743-24af-451e-b58d-81e3b727a3c8 tempest-VolumesNegativeTest-1800043371 tempest-VolumesNegativeTest-1800043371-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/3cf172dc-de43-4a1c-bcb8-28f272dec8cf" status: 404 len: 109 microversion: 2.1 time: 0.040576 Sep 11 00:54:07.302266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 86/162] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:54:07 2022] GET /compute/v2.1/os-volumes/3cf172dc-de43-4a1c-bcb8-28f272dec8cf => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:54:07.860356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:07.861138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:07.877479 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:07.877985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:07.878420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:07.879159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:07.879663 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:07.880093 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:08.118511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:08.128274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3d77ca15-30f2-447c-af09-1f8ac2a996ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.271889 Sep 11 00:54:08.129004 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 77/163] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:07 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:54:09.153189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:09.153890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:09.160559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:09.161117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:09.161617 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:09.162375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:09.162939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:09.163490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:09.310239 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:09.319948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e6b72812-ad19-42fe-bcad-8a55efc43fb9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.172755 Sep 11 00:54:09.320513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 87/164] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:09 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:10.343042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:10.343661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:10.346030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:10.352005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:10.352494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:10.352934 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:10.353628 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:10.354076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:10.354521 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:10.488083 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:10.496462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-91173495-90d0-4193-8eb3-0f6930a49c93 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1522 microversion: 2.1 time: 0.157526 Sep 11 00:54:10.497211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 78/165] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:10 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:11.520663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:11.521274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:11.527225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:11.527708 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:11.528144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:11.528841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:11.529291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:11.529711 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:11.678950 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:11.687940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fb3b86d8-05a2-4ae6-bc3e-42972304fe9d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1502 microversion: 2.1 time: 0.171038 Sep 11 00:54:11.688516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 88/166] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:11 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1502 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:11.712738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:11.714092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:11.724743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:11.725233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:11.726031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:11.727145 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:11.727758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:11.728373 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:11.872295 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:11.880086 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-46e746c7-cc81-4486-824b-b32b009cd783 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1516 microversion: 2.1 time: 0.172304 Sep 11 00:54:11.880644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 79/167] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:11 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1516 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:13.310603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:13.312891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:13.318721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:13.319279 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:13.319718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:13.320408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:13.320860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:13.321288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:13.325070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:13.343908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lazy-loading 'flavor' on Instance uuid 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:54:13.461563 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-32c18e5a-7521-4d93-93bf-4382f26d0f49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/action" status: 200 len: 32 microversion: 2.1 time: 0.156156 Sep 11 00:54:13.462165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 89/168] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:54:13 2022] POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/action => generated 32 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 11 00:54:13.472931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:13.473560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:13.480862 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:13.481441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:13.482005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:13.482765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:13.483285 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:13.483780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:13.624826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:13.634870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5c89d73a-5cc1-4b33-922c-cf053fa5032b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.166430 Sep 11 00:54:13.635622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 80/169] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:13 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:14.659304 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:14.660281 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:14.667930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:14.668428 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:14.668863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:14.669567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:14.670025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:14.670456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:14.821830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:14.829738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6b773bd7-98ba-476a-843e-21fcbcebda34 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.174721 Sep 11 00:54:14.830298 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 90/170] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:14 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:15.866592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:15.868074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:15.875636 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:15.876399 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:15.877003 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:15.885265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:15.886286 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:15.886958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:16.072275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:16.081532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f390d6ac-a9fe-4b18-a2db-0f31f19d4f46 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.219239 Sep 11 00:54:16.082404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 81/171] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:15 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:17.112856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:17.113466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:17.122465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:17.123122 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:17.123886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:17.124863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:17.125561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:17.126177 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:17.273548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:17.282836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-89196baa-4733-4168-b01e-ea77726a7eb6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.177169 Sep 11 00:54:17.295821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 91/172] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:17 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:18.310841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:18.312487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:18.321111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:18.321855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:18.322472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:18.324139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:18.324845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:18.325382 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:18.485582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:18.493254 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b5b3e0ce-bad8-4d72-9550-768c3b954de6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.189141 Sep 11 00:54:18.506810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 82/173] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:18 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:19.517627 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:19.518244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:19.520301 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=44 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:19.528631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:19.528631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:19.528631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:19.528631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:19.528956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:19.528956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:19.700063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:19.708916 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f1b74558-abda-4d73-80b7-825932aaca55 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.195063 Sep 11 00:54:19.719684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 92/174] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:19 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:20.737809 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:20.738630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:20.748474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:20.749357 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:20.750069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:20.751384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:20.752284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:20.753807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:21.161778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:21.171437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-80f4366e-440b-4ab3-b753-22b8fe8d39c2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.439206 Sep 11 00:54:21.172011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 83/175] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:20 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:22.208236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:22.208798 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:22.215561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:22.216055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:22.216494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:22.217479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:22.217940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:22.218707 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:22.421687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:22.429375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bcfc0d0d-612b-4f7b-974a-830e95c7795f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.225571 Sep 11 00:54:22.429945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 93/176] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:22 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:23.452928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:23.453542 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:23.460855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:23.461506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:23.462133 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:23.463969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:23.464844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:23.465487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:23.649065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:23.659569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f6a0d8a0-747d-45e5-b64b-595eb1603a53 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.210081 Sep 11 00:54:23.660840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 84/177] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:23 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:24.687375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:24.689105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:24.696017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:24.696758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:24.697399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:24.698479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:24.699160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:24.699854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:24.989987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:24.995627 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e61769d2-e964-4bd2-939e-186bf9d102d2 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.313582 Sep 11 00:54:24.998127 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 94/178] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:24 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:26.024604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:26.025211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:26.031741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:26.032226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:26.032659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:26.033346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:26.033799 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:26.034226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:26.169988 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:26.177674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-bc5c064a-f8dc-4924-9f69-97bdf70c48a9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.157152 Sep 11 00:54:26.178225 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 85/179] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:26 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:27.201059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:27.201671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:27.208046 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.208568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.209118 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.209979 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.210597 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.211134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.343290 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:27.351067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1723764a-ff8b-4155-82d1-9375880aeea4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1502 microversion: 2.1 time: 0.153918 Sep 11 00:54:27.351635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 95/180] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:27 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1502 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:27.375662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4622778f-1b1d-463e-a26e-bc2e26a1e87e"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:27.378044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:27.386099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.386666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.387129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.387826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.388281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.388705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.412744 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] HTTP exception thrown: Cannot 'attach_volume' instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 while it is in vm_state rescued Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Request method failure captured: Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/os-volume_attachments HTTP/1.1 Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 74 Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFZLkHjEmrxO7dxRA2K7NCkTkd6TbVWwXHlucfSpPoRvOHHB3oAJBaZcC7qeptnSWmQlNYC7VVp9rstzO1y1WIthC38SS1aTqw9V5bzg9Nair1V4TI7JOjqgjpvcNn4WH05oWOYYYTimbkYeWMgcOfVBndKoCvBDrkrzxnnHc0edIA Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:54:27.414320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_6a58aefc25e141ca93821a80859875e8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 5df6979effd84ece8d75b631ba7da196 Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {"volumeAttachment": {"volumeId": "4622778f-1b1d-463e-a26e-bc2e26a1e87e"}} Sep 11 00:54:27.415827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Cannot 'attach_volume' instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 while it is in vm_state rescued Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'server_id': '9acff3d4-2dd1-4809-bc64-2dfc7829bfc6', 'body': {'volumeAttachment': {'volumeId': '4622778f-1b1d-463e-a26e-bc2e26a1e87e'}}} Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:54:27.416785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 while it is in vm_state rescued Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:54:27.418191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 while it is in vm_state rescued Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Returning 409 to user: Cannot 'attach_volume' instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 while it is in vm_state rescued {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7b2e0607-33db-45ef-a812-2e596a358b7c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.044684 Sep 11 00:54:27.419449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 86/181] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:54:27 2022] POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/os-volume_attachments => generated 152 bytes in 46 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 11 00:54:27.425065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:27.425625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:27.431516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.432105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.432762 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.433442 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.433892 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.434310 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.504494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a761b778-fcf1-4d48-8ebe-335128721b65 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/action" status: 202 len: 0 microversion: 2.1 time: 0.083056 Sep 11 00:54:27.508127 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 96/182] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:54:27 2022] POST /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6/action => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 11 00:54:27.514319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:27.514969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:27.520955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.521435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.521901 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.522591 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:27.523234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:27.523615 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:27.658679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:27.667727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-49d046bc-5d52-4554-84d3-c59e3abe2c20 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1510 microversion: 2.1 time: 0.158225 Sep 11 00:54:27.668666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 87/183] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:27 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1510 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:54:28.693182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:28.693881 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:28.704288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:28.704288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:28.704288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:28.705283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:28.705283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:28.705283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:28.860364 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:28.871384 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3a158270-5a84-4aad-b50c-e722173fd2c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1510 microversion: 2.1 time: 0.182278 Sep 11 00:54:28.883854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 97/184] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:28 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1510 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:29.896329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:29.896941 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:29.904615 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:29.905114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:29.905529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:29.910124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:29.910124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:29.910124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:29.910371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=90 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:30.066039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:30.074941 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-53133160-6121-4709-a727-00358ce00c9c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1510 microversion: 2.1 time: 0.182380 Sep 11 00:54:30.075649 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 88/185] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:29 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1510 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:31.101278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:31.102705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:31.110564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:31.111312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:31.111915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:31.112967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:31.113577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:31.114170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:31.247775 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:31.255927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ee90ebde-1d2a-4a19-829a-eaed084dc73f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1510 microversion: 2.1 time: 0.158489 Sep 11 00:54:31.256590 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 98/186] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:31 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1510 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:32.280337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:32.280970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:32.288131 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:32.288621 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:32.289058 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:32.289754 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:32.290207 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:32.290642 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:32.434677 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:32.442429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-957e75eb-7a4e-4153-89d6-6cf4821c41f0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1516 microversion: 2.1 time: 0.166356 Sep 11 00:54:32.454970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 89/187] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:32 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1516 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:54:33.824902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4817c913-6363-45b8-9d56-36a71c7414df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'create', calling method: .version_select at 0x7fc01ad659d0>, body: {"keypair": {"name": "tempest-keypair-800080639", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFDZ5vJRRpSsrwpuKV5FiWuKGU9jyBBegXcIcWzD9E4uEe0IuhUnplSt1hWMp7EEuMien4G5jud8Uf8FZu9i2Fa4J5thXaV/flJymuiMMwYRzahyOjLRVWa4ZfA64z9pWg=="}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:33.840990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-4817c913-6363-45b8-9d56-36a71c7414df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for project 6a58aefc25e141ca93821a80859875e8. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:54:33.850771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-4817c913-6363-45b8-9d56-36a71c7414df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for user 5df6979effd84ece8d75b631ba7da196 and project 6a58aefc25e141ca93821a80859875e8. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:54:33.890941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4817c913-6363-45b8-9d56-36a71c7414df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.070191 Sep 11 00:54:33.891533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 99/188] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:54:33 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:54:35.468680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-886045069", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4d017d49-42d2-4f66-ab1e-6d7b9f9e874f"}], "security_groups": [{"name": "tempest-securitygroup--1390386321"}], "key_name": "tempest-keypair-800080639", "user_data": "}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:35.586247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] validate_networks() for [('4d017d49-42d2-4f66-ab1e-6d7b9f9e874f', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:54:36.283789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:54:36.284232 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:54:36.285159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:54:36.285723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:54:36.286126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:54:36.297515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:36.297965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:36.298584 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:36.299323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:36.300019 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:36.300437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:36.318164 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for project 6a58aefc25e141ca93821a80859875e8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:54:36.323838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Getting quotas for user 5df6979effd84ece8d75b631ba7da196 and project 6a58aefc25e141ca93821a80859875e8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:54:36.331340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:36.331973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:36.344352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:54:36.344555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:36.345102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:36.346528 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:36.347418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:54:36.347752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:36.348327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:36.348975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:36.390495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fab4b6c6-ca38-481b-882f-c6ff71560eb6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/ips" status: 200 len: 150 microversion: 2.1 time: 0.067120 Sep 11 00:54:36.392331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 100/189] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:54:36 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/ips => generated 150 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:54:36.404628 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-887608fa-4e70-4a69-be6a-23234d1ff28c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.940449 Sep 11 00:54:36.405712 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 90/190] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:54:35 2022] POST /compute/v2.1/servers => generated 409 bytes in 943 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:54:36.422907 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bae6ea78-8e9d-4603-98c6-e0ccc134db70 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:36.424348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bae6ea78-8e9d-4603-98c6-e0ccc134db70 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:36.513219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bae6ea78-8e9d-4603-98c6-e0ccc134db70 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:36.521999 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bae6ea78-8e9d-4603-98c6-e0ccc134db70 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1244 microversion: 2.1 time: 0.108826 Sep 11 00:54:36.524646 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 101/191] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:36 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1244 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:37.547441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:37.548889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:37.557429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:37.558180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:37.558685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:37.559687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:37.561659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:37.562408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:37.635729 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:37.654124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e39e3437-2fd7-4265-af85-ee25799917c0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1296 microversion: 2.1 time: 0.111367 Sep 11 00:54:37.656721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 91/192] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:37 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1296 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:38.125488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "193ee13b-53d8-4301-b1fd-7ec6fce1bdc7"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:38.129703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:38.143660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:38.144443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:38.145550 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:38.145994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:38.146815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:38.147419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:38.683331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:38.684614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:38.705574 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:38.707684 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:38.708150 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:38.712162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:38.714189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:38.715149 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:38.820733 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:38.832601 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-bc9d6435-94aa-4aa0-88e8-04c048da52e0 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1302 microversion: 2.1 time: 0.154843 Sep 11 00:54:38.833277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 92/193] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:38 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1302 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:38.881800 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-655e3f0f-0180-434a-ba8a-3dd771c0992d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.761045 Sep 11 00:54:38.883231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 102/194] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:54:38 2022] POST /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments => generated 194 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:54:39.859571 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:39.860194 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:39.866895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:39.867383 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:39.868035 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:39.869451 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:39.870114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:39.870703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:40.070044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:40.089149 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-989a4d89-dd68-4e03-a76b-5a1b001a1473 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.231439 Sep 11 00:54:40.090074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 93/195] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:39 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:40.440037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:40.441543 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:40.448717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:40.449207 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:40.449643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:40.450372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:40.450876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:40.451302 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:40.477417 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b7989bb0-bd02-459a-ba8f-c56d696343c4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/ips" status: 200 len: 151 microversion: 2.20 time: 0.042194 Sep 11 00:54:40.479085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 103/196] 10.222.0.27 () {60 vars in 1289 bytes} [Sun Sep 11 00:54:40 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/ips => generated 151 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:54:41.117476 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:41.120411 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:41.125178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=45,nova_api:INSERT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:41.129573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:41.130329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:41.131028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:41.131926 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:41.132601 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:41.133293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:41.311186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:41.319743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-37de966f-d9d7-4ea6-af90-5a73b467655b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.210528 Sep 11 00:54:41.336247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 94/197] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:41 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:42.346112 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:42.347158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:42.374329 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:42.378638 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:42.379217 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:42.380117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:42.380459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:42.380886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:42.386734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:42.545606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:42.555066 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7fbdcc1d-8428-4313-be16-6cc280fe1305 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.213405 Sep 11 00:54:42.555636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 104/198] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:42 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:42.841101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:42.842369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:42.854199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:42.854738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:42.855182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:42.856011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:42.856665 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:42.857258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:42.892287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'flavor' on Instance uuid 7ed280f0-7beb-47b6-a543-0e3eb7083067 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:54:43.180266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fe003ff1-f0b1-422c-9f87-366f0b186fca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/action" status: 202 len: 0 microversion: 2.20 time: 0.343911 Sep 11 00:54:43.184528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 95/199] 10.222.0.27 () {62 vars in 1319 bytes} [Sun Sep 11 00:54:42 2022] POST /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/action => generated 0 bytes in 346 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:54:43.190663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:43.191379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:43.198147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:43.198738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:43.199276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:43.200061 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:43.200612 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:43.201110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:43.217700 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-4fe19502-11a7-4a2f-8eaa-efa5c7212257 req-ba99d49d-e04a-49bb-8740-1d3830b9be25 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb3cc25c-09a9-47c9-bc2e-d5227478ab70", "tag": "f4ee5962-5a14-4994-83b5-3235d55d032f"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:43.239795 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4fe19502-11a7-4a2f-8eaa-efa5c7212257 req-ba99d49d-e04a-49bb-8740-1d3830b9be25 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:43.243111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4fe19502-11a7-4a2f-8eaa-efa5c7212257 req-ba99d49d-e04a-49bb-8740-1d3830b9be25 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:43.243111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4fe19502-11a7-4a2f-8eaa-efa5c7212257 req-ba99d49d-e04a-49bb-8740-1d3830b9be25 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:43.280936 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-4fe19502-11a7-4a2f-8eaa-efa5c7212257 req-ba99d49d-e04a-49bb-8740-1d3830b9be25 service nova] Creating event network-changed:f4ee5962-5a14-4994-83b5-3235d55d032f for instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 on hv-cinder-maste Sep 11 00:54:43.288221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-4fe19502-11a7-4a2f-8eaa-efa5c7212257 req-ba99d49d-e04a-49bb-8740-1d3830b9be25 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074715 Sep 11 00:54:43.295019 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 96/200] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:54:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:54:43.414760 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:43.426505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:43.428340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:43.429954 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c43c8464-0f3d-4819-a71e-353683c4f3a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.244020 Sep 11 00:54:43.431807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 105/201] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:43 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:54:43.578500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:43.579270 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:43.594716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:43.595486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:43.596119 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:43.596985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:43.597595 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:43.598187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:43.814281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:43.822662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4532888f-f422-4046-a92d-02bb7c27c8de tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.247766 Sep 11 00:54:43.823910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 97/202] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:43 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:44.459332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:44.460886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:44.469776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:44.476814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:44.477704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:44.478758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:44.479350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:44.479681 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:44.737235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:44.753597 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:44.755133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:44.761094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-55d756b9-776d-47b0-b018-1045c873c9e4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.304252 Sep 11 00:54:44.775644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 106/203] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:44 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:44.848389 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:44.849627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:44.862134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:44.862740 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:44.863484 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:44.864715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:44.866184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:44.866900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:45.170692 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:45.191082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-76cfb6f2-fb0e-41e3-859d-05a6c4585bd1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.347471 Sep 11 00:54:45.192796 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 98/204] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:44 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:54:45.800348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:45.801499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:45.818913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:45.819728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:45.820873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:45.822089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:45.822847 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:45.824228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:46.116639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:46.128485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:46.130526 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:46.135472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a5b93c6d-fe42-4710-8f2d-52473c310c63 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.350324 Sep 11 00:54:46.155397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 107/205] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:45 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:46.219270 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:46.219915 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:46.238520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:46.239051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:46.239490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:46.240200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:46.240652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:46.241092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:46.318110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:46.336907 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:46.601494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:46.639994 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5a1f5f5c-85ea-4fb8-af39-d02d7f1b8027 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.426495 Sep 11 00:54:46.641201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 99/206] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:46 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:47.086432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:47.164748 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:47.167052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:47.176977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:47.178280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:47.178911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:47.179844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:47.180446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:47.181581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:47.350138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:47.360940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:47.362732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:47.364045 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2abd204a-170f-4be5-9a21-947d833c142c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.205031 Sep 11 00:54:47.365029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 100/207] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:47 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:54:47.370630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-085b4fe4-5629-472a-817e-144e47eb9ab0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Action: 'create', calling method: .version_select at 0x7fc01acf7040>, body: {"keypair": {"name": "tempest-TestShelveInstance-1739406180", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIuloizDJcVEIpHaE46eh8QzUUG/9u2glcYY1dzShMe2wEI6qFwT/UnVYiWufQEDlU0N90xu0itGXjDN+HLnHg8Inm7Vv5vn7QH7gFQ5e5YwFzwBbEpEoM2ikpph2u3d9w=="}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:47.388266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-085b4fe4-5629-472a-817e-144e47eb9ab0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Getting quotas for project d55d41063ec34fb6921c367025fa9af9. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:54:47.393183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-085b4fe4-5629-472a-817e-144e47eb9ab0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Getting quotas for user 967e0ef4e6364804a58d0d59c22cd6a8 and project d55d41063ec34fb6921c367025fa9af9. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:54:47.419271 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-085b4fe4-5629-472a-817e-144e47eb9ab0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.337379 Sep 11 00:54:47.419956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 108/208] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:54:47 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:54:47.663659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:47.664374 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:47.673080 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:47.673826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:47.674479 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:47.675282 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:47.675909 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:47.676338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:47.820848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:47.838029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fa98233d-d4b9-4ad1-828e-406f476274ff tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.177572 Sep 11 00:54:47.840110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 101/209] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:47 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:48.390555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:48.391858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:48.402603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:48.404810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:48.405682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:48.407672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:48.408320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:48.409214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:48.622603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:48.636822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:48.639789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:48.643785 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b5719a0f-4ca6-45fa-a8ab-e3a453804f18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.257394 Sep 11 00:54:48.645133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 109/210] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:48 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:48.867438 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:48.869739 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:48.878922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:48.879883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:48.880546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:48.881854 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:48.882572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:48.883492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:49.088958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:49.107805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-dff26a73-d81b-4943-89fa-62d46175d5f8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.249863 Sep 11 00:54:49.109629 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 102/211] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:48 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:49.673634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:49.674224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:49.682716 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:49.688320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:49.689237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:49.690309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:49.703315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:49.704298 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:49.704943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:50.060836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:50.072396 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:50.078941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:50.089359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-eb273ad1-709d-496a-a241-f4b757fadfa6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.419632 Sep 11 00:54:50.090684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 110/212] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:49 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:50.136105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:50.136726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:50.147118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:50.152651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:50.153130 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:50.154112 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:50.154592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:50.155072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:50.788470 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:50.798836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-51d30284-8e71-4692-8fe6-e30f6493c98f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.668739 Sep 11 00:54:50.799428 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 103/213] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:50 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:51.126686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:51.127450 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:51.134993 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:51.135786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:51.136300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:51.137185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:51.137736 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:51.138232 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:51.456105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:51.469824 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:51.476168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:51.478764 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5a027125-1483-4652-9f01-6446b1ed795f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.358412 Sep 11 00:54:51.479798 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 111/214] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:51 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:51.823155 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:51.823858 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:51.834166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:51.834668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:51.835135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:51.836326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:51.836786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:51.837783 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:51.875939 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:52.140237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:52.152718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9aca86b5-01f6-4703-a3df-3810ee583e1b tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.333271 Sep 11 00:54:52.153301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 104/215] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:51 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:52.505709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:52.507497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:52.518780 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:52.521730 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:52.524136 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:52.526087 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:52.526969 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:52.527571 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:52.635682 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3ce4084d-b78d-4c68-882b-c536cec6e118 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: .version_select at 0x7fc01aea2ee0>, body: {"keypair": {"name": "tempest-keypair-1845430137", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPQpfEx24QaBtG7rQ3MWfHE+QHQZhpEH8etJ4F+Wkq5LeAuF/hMllGzTDV7ffLuivPvtFrBmK5exFfDh9VbCH9SUUKBq02ERjzsYKVlZ2vxKmWKIbJXD4pFg75ewo35guw=="}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:52.667691 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-3ce4084d-b78d-4c68-882b-c536cec6e118 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:54:52.683328 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-3ce4084d-b78d-4c68-882b-c536cec6e118 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:54:52.748728 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3ce4084d-b78d-4c68-882b-c536cec6e118 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.117196 Sep 11 00:54:52.749244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 105/216] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:54:52 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:54:52.799608 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:52.811991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:52.818768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:52.820515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e959658d-3d15-476d-9645-a24137ab7652 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.318540 Sep 11 00:54:52.822255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 112/217] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:52 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:53.176548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:53.177177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:53.183518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:53.184242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:53.184837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:53.185705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:53.186346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:53.186959 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:53.466420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:53.479717 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-49ea3c66-7d30-4167-92b0-373b114b5ff1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.305285 Sep 11 00:54:53.497838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 106/218] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:53 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:53.854763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:53.857051 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:53.871311 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:53.872641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:53.881717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:53.886472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:53.887721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:53.891506 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:54.327552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:54.341211 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:54.343116 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:54.344371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6f253010-a3bd-4416-af43-a58ad47e0115 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.496457 Sep 11 00:54:54.344951 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 113/219] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:53 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:54.508062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:54.511664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:54.520981 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:54.521884 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:54.522520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:54.523420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:54.524050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:54.524737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:55.028942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:55.042635 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cd4631fd-eec8-4c95-9049-3a335a5b250e tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.540359 Sep 11 00:54:55.059234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 107/220] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:54 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:55.324080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1495083536", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e5ef326a-1def-4c9c-ad55-20a6d9fdbde1"}], "security_groups": [{"name": "tempest-securitygroup--1141183280"}], "key_name": "tempest-keypair-1845430137", "user_data": "}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:55.381137 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:55.381986 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:55.406915 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:55.408473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:55.409680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:55.410588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:55.411226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:55.411818 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:55.610256 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] validate_networks() for [('e5ef326a-1def-4c9c-ad55-20a6d9fdbde1', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:54:55.989157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:56.010606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:56.012106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:56.013298 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7b26da2e-8119-46c9-b60f-470a52c1f961 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.637538 Sep 11 00:54:56.029342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 108/221] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:55 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:54:56.035076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-4d6b2405-dcd4-4dc5-89ee-e77220099d44 req-4aa2830f-db2f-4a18-b236-3f8c6c4a2fa1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb3cc25c-09a9-47c9-bc2e-d5227478ab70", "name": "network-vif-plugged", "status": "completed", "tag": "f4ee5962-5a14-4994-83b5-3235d55d032f"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:54:56.045436 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4d6b2405-dcd4-4dc5-89ee-e77220099d44 req-4aa2830f-db2f-4a18-b236-3f8c6c4a2fa1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:56.046066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4d6b2405-dcd4-4dc5-89ee-e77220099d44 req-4aa2830f-db2f-4a18-b236-3f8c6c4a2fa1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:56.046563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4d6b2405-dcd4-4dc5-89ee-e77220099d44 req-4aa2830f-db2f-4a18-b236-3f8c6c4a2fa1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:56.064910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-4d6b2405-dcd4-4dc5-89ee-e77220099d44 req-4aa2830f-db2f-4a18-b236-3f8c6c4a2fa1 service nova] Creating event network-vif-plugged:f4ee5962-5a14-4994-83b5-3235d55d032f for instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 on hv-cinder-maste Sep 11 00:54:56.070222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-4d6b2405-dcd4-4dc5-89ee-e77220099d44 req-4aa2830f-db2f-4a18-b236-3f8c6c4a2fa1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039401 Sep 11 00:54:56.070823 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 109/222] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:54:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:54:56.077024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:56.077735 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:56.084127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:56.084689 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:56.085046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:56.085754 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:56.086208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:56.086632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:56.726217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:56.735193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4b1e38be-7ade-43fd-a281-93f2f69884d4 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.662399 Sep 11 00:54:56.735773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 110/223] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:56 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:56.841008 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:54:56.841503 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:54:56.842326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:54:56.842755 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:54:56.843201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:54:56.854569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:56.855194 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:56.855657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:56.856747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:56.857261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:56.857687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:56.877706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:54:56.882100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:54:56.894996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:54:56.897752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:54:56.975178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-77506e81-f411-4064-8c46-75d263f1e762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.660287 Sep 11 00:54:56.976160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 114/224] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:54:55 2022] POST /compute/v2.1/servers => generated 402 bytes in 1662 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:54:56.988778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f083ff60-d320-4757-9d50-8c293484855f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:56.995658 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f083ff60-d320-4757-9d50-8c293484855f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:57.041213 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:57.042914 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:57.050981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:57.056511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:57.057006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:57.059599 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:57.062131 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:57.062602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:57.070511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f083ff60-d320-4757-9d50-8c293484855f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:57.082667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f083ff60-d320-4757-9d50-8c293484855f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1242 microversion: 2.1 time: 0.098137 Sep 11 00:54:57.083539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 111/225] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:56 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1242 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:57.335069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:57.338442 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:57.339930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:57.341110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a7b3cac0-4830-4708-9fa6-e5db3a320f8f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.305693 Sep 11 00:54:57.346255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 115/226] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:57 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:54:57.768044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:57.769214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:57.782590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:57.783764 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:57.784714 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:57.785670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:57.786349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:57.787033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:58.106581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:58.116297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:58.117498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:58.129072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:58.131477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ee93b220-7176-4ed1-9614-cae26e0413ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1370 microversion: 2.1 time: 0.371004 Sep 11 00:54:58.132550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 112/227] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:57 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1370 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:58.133297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:58.133945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:58.134873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:58.135489 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:58.136075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:58.217598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:58.226291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-158c9f02-d98a-4aaf-9508-6232e224af6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1246 microversion: 2.1 time: 0.117891 Sep 11 00:54:58.228184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 116/228] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:58 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1246 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:58.393521 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:58.394905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:58.412531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:58.413354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:58.414094 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:58.416296 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:58.416977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:58.417588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:58.771151 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:58.786244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:58.788836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:58.790996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-10a9867f-03d6-4b18-9155-c9ae1a0129e1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.420340 Sep 11 00:54:58.792436 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 113/229] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:58 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:54:59.159276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:59.160619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:59.181043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.184545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.185228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.189157 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.191448 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.195963 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.252907 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:59.255317 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:59.270264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.271084 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.271538 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.272256 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.272718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.273151 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.518691 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:59.544457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1e7cebcf-8ebf-4e04-8b0d-a56bed7eace5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1294 microversion: 2.1 time: 0.295862 Sep 11 00:54:59.545479 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 114/230] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:59 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1294 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:54:59.638327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:54:59.657663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3c7b0feb-182d-45da-a2a3-ce11fa01e0fc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1564 microversion: 2.1 time: 0.505359 Sep 11 00:54:59.658255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 117/231] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:54:59 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1564 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:54:59.687880 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:59.689449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:59.704664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.705511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.706283 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.707456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.708231 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.708830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.819728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:54:59.820579 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:54:59.827931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.828756 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.829402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.830397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:54:59.831097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:54:59.832133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:54:59.989170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a5ffc84d-543a-450e-b9fc-36f809f47edf tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.310414 Sep 11 00:54:59.998169 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 115/232] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 00:54:59 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-interface => generated 284 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:00.345175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:00.361473 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:00.364406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:00.371003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7f75c950-9da4-4325-9dc5-bba65abbcc93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.556479 Sep 11 00:55:00.372187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 118/233] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:54:59 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:00.569321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:00.570764 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:00.581011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:00.581886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:00.582718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:00.583672 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:00.584345 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:00.584989 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:00.746120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:00.773819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2a7ea944-fdc4-43d2-a507-c8f50a93ccbc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1300 microversion: 2.1 time: 0.208272 Sep 11 00:55:00.779026 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 116/234] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:00 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1300 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:01.406194 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:01.407622 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:01.425799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:01.426569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:01.427272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:01.428286 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:01.429509 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:01.431807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:01.805955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:01.808529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:01.825724 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:01.826781 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:01.828863 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:01.830110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:01.831957 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:01.834040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:02.411555 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:02.453695 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f28e3185-563f-418e-824d-326279259680 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1300 microversion: 2.1 time: 0.655334 Sep 11 00:55:02.455161 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 117/235] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:01 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1300 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:02.681079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:02.696183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:02.701887 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:02.705931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a19e20e1-5275-403f-ae8c-89de74e1f0ea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1743 microversion: 2.20 time: 1.307517 Sep 11 00:55:02.711099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 119/236] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:01 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1743 bytes in 1314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:03.480475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:03.482020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:03.497798 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:03.498299 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:03.499777 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:03.500740 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:03.502581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:03.503680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:03.717201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-3cc77122-c8aa-45d7-911b-32ee0dca7a0c req-dcf5956f-4af3-44c4-b446-a20df1deed23 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb3cc25c-09a9-47c9-bc2e-d5227478ab70", "tag": "f4ee5962-5a14-4994-83b5-3235d55d032f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:03.739862 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3cc77122-c8aa-45d7-911b-32ee0dca7a0c req-dcf5956f-4af3-44c4-b446-a20df1deed23 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:03.740294 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3cc77122-c8aa-45d7-911b-32ee0dca7a0c req-dcf5956f-4af3-44c4-b446-a20df1deed23 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:03.740935 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3cc77122-c8aa-45d7-911b-32ee0dca7a0c req-dcf5956f-4af3-44c4-b446-a20df1deed23 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:03.775588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-3cc77122-c8aa-45d7-911b-32ee0dca7a0c req-dcf5956f-4af3-44c4-b446-a20df1deed23 service nova] Creating event network-changed:f4ee5962-5a14-4994-83b5-3235d55d032f for instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 on hv-cinder-maste Sep 11 00:55:03.796438 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-3cc77122-c8aa-45d7-911b-32ee0dca7a0c req-dcf5956f-4af3-44c4-b446-a20df1deed23 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085425 Sep 11 00:55:03.802113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 120/237] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:55:03.814698 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:03.815454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:03.834121 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:03.834628 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:03.835101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:03.848860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:03.849476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:03.849939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:04.268789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:04.292307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c6ad72e9-f197-436d-aeab-139b061d7e97 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.818109 Sep 11 00:55:04.310970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 118/238] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:03 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:04.412434 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:04.430871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:04.432980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:04.434331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ca76b7f7-9c07-4569-a043-533975afa019 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.630412 Sep 11 00:55:04.435948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 121/239] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:03 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:05.317835 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:05.319216 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:05.332513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:05.339662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:05.345575 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:05.350758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:05.351623 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:05.352859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:05.472877 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:05.475344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:05.491922 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:05.493023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:05.493754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:05.494714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:05.495425 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:05.499193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:06.159713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:06.173236 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-029a9487-947f-429a-a31f-f50204ae215a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.859315 Sep 11 00:55:06.182457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 119/240] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:05 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 869 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:06.217545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:06.229619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:06.240799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:06.242315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-986837ab-bbba-4aa6-9902-53855e5dfe42 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.775847 Sep 11 00:55:06.243644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 122/241] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:05 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 778 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:06.313485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-a08c55ec-8710-4c23-a789-8c61f794ebbc req-b978ccdc-6215-4894-ac86-4806375cd6bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a843e04-0782-4e1e-b379-9be9a7106442", "tag": "d120b4b9-10bc-4478-8187-9b7014b08cef"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:06.345057 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a08c55ec-8710-4c23-a789-8c61f794ebbc req-b978ccdc-6215-4894-ac86-4806375cd6bd service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:06.355486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a08c55ec-8710-4c23-a789-8c61f794ebbc req-b978ccdc-6215-4894-ac86-4806375cd6bd service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:06.356171 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a08c55ec-8710-4c23-a789-8c61f794ebbc req-b978ccdc-6215-4894-ac86-4806375cd6bd service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:06.364797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-2044789397", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1739406180", "security_groups": [{"name": "tempest-secgroup-smoke-1981775819"}], "networks": [{"uuid": "4e359fd9-06a0-4943-a4a2-16a223ad6963"}], "block_device_mapping_v2": [{"uuid": "d428e359-95f3-474f-8984-947d87d222af", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:06.402396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-a08c55ec-8710-4c23-a789-8c61f794ebbc req-b978ccdc-6215-4894-ac86-4806375cd6bd service nova] Creating event network-changed:d120b4b9-10bc-4478-8187-9b7014b08cef for instance 9a843e04-0782-4e1e-b379-9be9a7106442 on hv-cinder-maste Sep 11 00:55:06.427494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-a08c55ec-8710-4c23-a789-8c61f794ebbc req-b978ccdc-6215-4894-ac86-4806375cd6bd service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.117701 Sep 11 00:55:06.430287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 120/242] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:06.624870 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] validate_networks() for [('4e359fd9-06a0-4943-a4a2-16a223ad6963', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:55:06.872886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:06.891196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=10 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:07.213651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:07.214548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:07.224105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:07.227776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:07.228666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:07.229397 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:07.230372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:07.231215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:07.231891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:07.734889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:55:07.736010 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:55:07.736664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:55:07.737333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:55:07.737914 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:55:07.758636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:07.759662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:07.760307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:07.761281 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:07.761922 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:07.762518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:07.777567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:07.784618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:07.794154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7170377b-9b45-4015-9f53-9bb5062f8675 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.589090 Sep 11 00:55:07.797351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 121/243] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:07 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:07.809710 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:07.810594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:07.816992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Getting quotas for project d55d41063ec34fb6921c367025fa9af9. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:55:07.831941 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:07.833597 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:07.837023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Getting quotas for user 967e0ef4e6364804a58d0d59c22cd6a8 and project d55d41063ec34fb6921c367025fa9af9. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:55:07.837363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:07.838302 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:07.838991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:07.839923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:08.057247 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:55:08.064088 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d428e359-95f3-474f-8984-947d87d222af',volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:55:08.282696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:08.299313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:08.305244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:08.308567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-78b76568-6d32-437f-baed-0bceaf71db45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.509422 Sep 11 00:55:08.310273 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 122/244] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:07 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:08.388829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-f03f2908-809f-4501-a8e0-2f8cbcbce62b req-2e3b8009-0b12-4c67-952a-486c44fb2747 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "name": "network-vif-plugged", "status": "completed", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:08.459796 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f03f2908-809f-4501-a8e0-2f8cbcbce62b req-2e3b8009-0b12-4c67-952a-486c44fb2747 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:08.460681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f03f2908-809f-4501-a8e0-2f8cbcbce62b req-2e3b8009-0b12-4c67-952a-486c44fb2747 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:08.462234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f03f2908-809f-4501-a8e0-2f8cbcbce62b req-2e3b8009-0b12-4c67-952a-486c44fb2747 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:08.483074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-f03f2908-809f-4501-a8e0-2f8cbcbce62b req-2e3b8009-0b12-4c67-952a-486c44fb2747 service nova] Creating event network-vif-plugged:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:55:08.489200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c94f77c6-9085-44fc-8f1f-f7626739cc8f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 2.128240 Sep 11 00:55:08.493188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-f03f2908-809f-4501-a8e0-2f8cbcbce62b req-2e3b8009-0b12-4c67-952a-486c44fb2747 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.145861 Sep 11 00:55:08.494982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 123/245] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:08.500984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 123/246] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:55:06 2022] POST /compute/v2.1/servers => generated 406 bytes in 2141 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:55:08.521892 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f0d19e94-b1e7-4fd5-bce9-1d0f52d66021 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:08.523011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f0d19e94-b1e7-4fd5-bce9-1d0f52d66021 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:08.645386 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f0d19e94-b1e7-4fd5-bce9-1d0f52d66021 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:08.653798 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f0d19e94-b1e7-4fd5-bce9-1d0f52d66021 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1089 microversion: 2.1 time: 0.137787 Sep 11 00:55:08.656582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 124/247] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:08 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1089 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:08.826005 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:08.827006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:08.834848 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:08.846019 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:08.847656 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:08.850764 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:08.851422 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:08.857996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:09.135538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:09.154986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8d7aaaf1-9f7b-4d71-9e38-ff0b43b28517 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.332900 Sep 11 00:55:09.156879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 124/248] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:08 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:09.338823 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:09.340853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:09.348979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:09.354233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:09.355074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:09.356053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:09.356731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:09.357389 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:09.678126 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9cda8b55-e4d6-42ca-a6f7-ddacfffc27f0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:09.678952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9cda8b55-e4d6-42ca-a6f7-ddacfffc27f0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:09.798766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:09.831434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:09.838645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:09.843030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6b679767-484e-4f5c-a039-6b1b10379ccc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.508916 Sep 11 00:55:09.862819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 125/249] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:09 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:09.906468 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9cda8b55-e4d6-42ca-a6f7-ddacfffc27f0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:09.924487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9cda8b55-e4d6-42ca-a6f7-ddacfffc27f0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1089 microversion: 2.1 time: 0.248702 Sep 11 00:55:09.941785 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 125/250] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:09 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1089 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:10.186527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:10.187239 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:10.198965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:10.199794 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:10.200400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:10.202133 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:10.203369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:10.205152 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:10.221566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=100 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:10.621902 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:10.649931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3343059f-8819-4c56-8ff5-3c6cba070c56 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.463893 Sep 11 00:55:10.656188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 126/251] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:10 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:10.876188 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:10.876807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:10.888923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:10.889336 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:10.889770 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:10.890485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:10.906556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:10.909838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:10.956490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:10.957671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:10.968182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:10.971711 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:10.972368 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:10.973263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:10.973876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:10.975390 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:11.103261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:11.118871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-171338ea-6dbe-488a-87d2-3273979a14c7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1187 microversion: 2.1 time: 0.166198 Sep 11 00:55:11.123938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 127/252] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:10 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1187 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:11.252883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:11.270590 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:11.284845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:11.286279 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4e1f8320-ba8d-4945-a226-c0a9df54df66 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.414187 Sep 11 00:55:11.287568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 126/253] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:10 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:11.683947 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:11.685301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:11.757454 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:11.763882 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:11.766200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:11.768403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:11.771279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:11.776797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:11.777493 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:12.154774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:12.156392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:12.175516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:12.193542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:12.195467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:12.198949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:12.200349 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:12.205166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:12.249081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:12.271816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b924dca9-10ce-402d-a20f-67f86fb4172d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.591851 Sep 11 00:55:12.275847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 128/254] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:11 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:12.318535 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:12.319588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:12.327896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:12.331126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:12.332811 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:12.334177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:12.338162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:12.340673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:12.408008 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:12.417547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fb9038ed-6dc5-4323-9cc7-9d750df624a2 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1205 microversion: 2.1 time: 0.268557 Sep 11 00:55:12.439466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 127/255] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:12 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1205 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:12.779312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:12.790914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:12.792485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:12.793754 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9c184d65-7919-41d7-95f2-4d5f593df6b6 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.481688 Sep 11 00:55:12.803098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 129/256] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:12 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:13.297948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:13.298693 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:13.309952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:13.313600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:13.314166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:13.315065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:13.317993 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:13.318685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:13.455586 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:13.457332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:13.470255 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:13.470755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:13.487000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:13.491257 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:13.491976 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:13.496313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:13.733233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:13.742850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:13.752021 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-179bc4a4-4e08-48cd-89a3-30532ace5183 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1193 microversion: 2.1 time: 0.302295 Sep 11 00:55:13.752617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 130/257] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:13 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1193 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:13.773145 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fe8ff772-8eec-46f3-898a-c1ce2de3aea9 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.479527 Sep 11 00:55:13.781337 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 128/258] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:13 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:13.822417 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:13.823081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:13.843488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:13.844372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:13.845006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:13.852191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:13.853335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:13.853991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:14.204161 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:14.213294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:14.224121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:14.225870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-885e6d64-f5a8-43a5-a47b-9071af8f4088 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.408988 Sep 11 00:55:14.226726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 131/259] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:13 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:14.778594 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:14.779240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:14.793291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:14.800778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:14.801321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:14.802081 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:14.809047 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:14.810017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:14.810931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:14.813000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:14.821539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:14.822374 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:14.823064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:14.824493 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:14.825343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:14.825999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:15.246279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:15.268334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:15.285916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ca8cb3d5-4ff5-4125-a635-6270dcbee2e0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.480571 Sep 11 00:55:15.288623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1b2eaacf-25dd-4f40-acf5-8c354c750332 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.514809 Sep 11 00:55:15.294557 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 129/260] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:14 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:15.296688 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 132/261] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:14 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:15.311705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:15.313248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:15.321312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:15.322181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:15.322671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:15.323690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:15.325431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:15.326746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:16.243274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-c2f24f00-2caa-4415-9322-cc77c19b501c req-00c3e6c9-1a0e-4fa3-9215-6982429d266d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:16.262708 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c2f24f00-2caa-4415-9322-cc77c19b501c req-00c3e6c9-1a0e-4fa3-9215-6982429d266d service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:16.266118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c2f24f00-2caa-4415-9322-cc77c19b501c req-00c3e6c9-1a0e-4fa3-9215-6982429d266d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:16.270869 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c2f24f00-2caa-4415-9322-cc77c19b501c req-00c3e6c9-1a0e-4fa3-9215-6982429d266d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:16.294389 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-c2f24f00-2caa-4415-9322-cc77c19b501c req-00c3e6c9-1a0e-4fa3-9215-6982429d266d service nova] Creating event network-changed:549957cc-ef50-4e12-9608-80b215c4bfb4 for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c on hv-cinder-maste Sep 11 00:55:16.317294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-c2f24f00-2caa-4415-9322-cc77c19b501c req-00c3e6c9-1a0e-4fa3-9215-6982429d266d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078323 Sep 11 00:55:16.319271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 133/262] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:55:16.319633 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:16.329555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:16.336683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:16.338903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8784eb25-5f47-45d1-ae39-28b3ff99f9a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 1.042917 Sep 11 00:55:16.341149 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:16.343670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:16.368696 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 130/263] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:15 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 1064 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:16.370911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:16.374651 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:16.375451 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:16.376181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:16.377156 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:16.377987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:16.378662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:16.387644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:16.406396 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:16.411524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:16.414625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:16.415588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:16.422416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:16.423552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:16.814718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:16.827075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-214a8463-0245-4f69-984e-ea4762ac9357 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.498784 Sep 11 00:55:16.830700 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 134/264] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:16 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:16.973137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:16.990636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e915e8dc-a69d-4bdf-9885-ff5404c46ff1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.620241 Sep 11 00:55:16.991277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 131/265] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:16 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:17.378531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:17.379943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:17.394602 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:17.395633 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:17.396336 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:17.397303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:17.397977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:17.398659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:17.712121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:17.723138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:17.724678 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:17.728188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-77e58ace-175f-478d-95f1-78a1fb29a510 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.353530 Sep 11 00:55:17.750819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 135/266] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:17 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:17.858701 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:17.859986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:17.862167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:17.882897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:17.891516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:17.895071 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:17.896173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:17.896873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:17.897533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:18.022172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:18.023139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:18.040777 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:18.041783 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:18.042703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:18.043681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:18.044382 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:18.045033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:18.500094 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:18.530492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4cd260c1-c3f0-4080-935e-c0ccdadbbee3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.497701 Sep 11 00:55:18.531582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 136/267] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:18 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:18.537465 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-57444ab0-f794-40d9-a996-7489a3716a5c req-8b4fcb0a-b208-4f48-811a-dd43126ad2ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a843e04-0782-4e1e-b379-9be9a7106442", "name": "network-vif-plugged", "status": "completed", "tag": "d120b4b9-10bc-4478-8187-9b7014b08cef"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:18.565853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-57444ab0-f794-40d9-a996-7489a3716a5c req-8b4fcb0a-b208-4f48-811a-dd43126ad2ba service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:18.572737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-57444ab0-f794-40d9-a996-7489a3716a5c req-8b4fcb0a-b208-4f48-811a-dd43126ad2ba service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:18.578400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-57444ab0-f794-40d9-a996-7489a3716a5c req-8b4fcb0a-b208-4f48-811a-dd43126ad2ba service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:18.606241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:18.609424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-57444ab0-f794-40d9-a996-7489a3716a5c req-8b4fcb0a-b208-4f48-811a-dd43126ad2ba service nova] Creating event network-vif-plugged:d120b4b9-10bc-4478-8187-9b7014b08cef for instance 9a843e04-0782-4e1e-b379-9be9a7106442 on hv-cinder-maste Sep 11 00:55:18.619669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-57444ab0-f794-40d9-a996-7489a3716a5c req-8b4fcb0a-b208-4f48-811a-dd43126ad2ba service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086591 Sep 11 00:55:18.627055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 137/268] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:18.631023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-29b77b8e-0c86-4e1e-a696-d5d85161a39f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.779902 Sep 11 00:55:18.637687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 132/269] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:17 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:18.762778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:18.763659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:18.791143 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:18.793239 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:18.799192 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:18.800031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:18.801062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:18.801756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:18.802416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:19.455840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:19.489128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:19.515995 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:19.532296 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-22e944de-5ad6-4087-a393-65a1725751bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.762466 Sep 11 00:55:19.535443 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 138/270] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:18 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 778 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:19.562865 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:19.563753 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:19.578317 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:19.586442 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:19.589398 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:19.597608 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:19.599496 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:19.600693 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:19.662365 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:19.663065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:19.693066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:19.693577 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:19.694077 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:19.694897 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:19.695500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:19.695937 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:19.815244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:19.837008 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ad88c1cf-27cf-427f-88b8-6e87d768475c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.281959 Sep 11 00:55:19.842186 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 133/271] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:19 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:19.982742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:20.015651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-48840914-29c9-4c61-896e-df848141d780 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.357845 Sep 11 00:55:20.017057 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 139/272] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:19 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:20.569259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:20.570860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:20.573743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=8 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:20.577542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=76,nova_api:INSERT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:20.603270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:20.604058 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:20.604679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:20.605717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:20.606391 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:20.609293 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:20.865533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:20.866154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:20.880180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:20.881201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:20.881825 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:20.887805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:20.888560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:20.889159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:20.890147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:20.901459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:20.908976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:20.911106 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0539b8d9-2c3b-4ee6-a490-201a707e2924 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.348152 Sep 11 00:55:20.927535 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 134/273] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:20 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:21.038706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:21.039577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:21.047938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:21.051057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:21.054285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:21.055982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:21.056686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:21.065238 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:21.343656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:21.362464 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-32c16e7f-f52e-4ad5-a4e7-ed03eb246762 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.498808 Sep 11 00:55:21.363791 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 140/274] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:20 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:21.542054 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:21.563426 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ba71d433-5a27-4c59-9bf4-dbeaedd5d895 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.527553 Sep 11 00:55:21.565101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 135/275] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:21 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:21.944022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:21.945829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:21.984391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:21.984849 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:21.985310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:21.995726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:21.996226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:21.996668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:22.392111 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:22.392755 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:22.418417 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:22.418912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:22.419377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:22.420121 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:22.422782 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:22.423288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:22.481015 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:22.494996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:22.507880 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:22.520370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-119b9afe-b97c-42c2-8c80-ca32a01ddac3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.580558 Sep 11 00:55:22.535427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 141/276] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:21 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:22.591634 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:22.592262 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:22.605590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:22.606427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:22.607065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:22.607958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:22.611573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:22.612204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:22.746212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:22.754392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8c70a193-d3f2-4227-8ca2-d336f40b9285 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1368 microversion: 2.1 time: 0.368668 Sep 11 00:55:22.756347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 136/277] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:22 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1368 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:22.971846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:22.990012 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-14c5dde2-40f3-4eeb-9555-845cb90c07d1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.402422 Sep 11 00:55:22.991282 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 142/278] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:22 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:23.553965 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:23.556151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:23.573238 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:23.573741 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:23.574177 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:23.574919 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:23.575378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:23.575800 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:23.782492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:23.783400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:23.798060 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:23.810360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:23.812755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:23.817127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:23.818945 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:23.820845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:24.111463 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:24.125749 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:24.128613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:24.130050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0369c3a6-f8cb-4d53-aca0-0525f3b7d01c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.581015 Sep 11 00:55:24.137195 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 137/279] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:23 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:24.146508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:24.147362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:24.175234 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:24.177346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:24.180648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:24.184581 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:24.187476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:24.188192 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:24.228410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:24.237517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f17a6912-2824-4a7d-a704-2ece48456919 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1558 microversion: 2.1 time: 0.459257 Sep 11 00:55:24.238982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 143/280] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:23 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1558 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:24.263281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:24.264183 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:24.281363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:24.285267 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:24.285940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:24.287229 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:24.302765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:24.304037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:24.315209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=100 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:24.596252 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:24.609307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3fe21600-765a-444c-9cd5-457c2dd3f132 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.462236 Sep 11 00:55:24.610615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 138/281] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:24 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:24.634746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-566803a5-0ba5-4dae-981e-07bb0ae014c2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.375296 Sep 11 00:55:24.635736 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 144/282] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 00:55:24 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/os-interface => generated 284 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:55:25.180810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:25.181961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:25.202338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:25.208645 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:25.209521 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:25.211285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:25.214649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:25.222812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:25.632149 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:25.632992 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:25.647753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:25.655587 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:25.658659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:25.660553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:25.661187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:25.664968 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:25.930368 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:25.938745 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:25.940229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:25.953312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-788fb4e5-0f74-40d3-a1f8-9f942d0ac742 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.776633 Sep 11 00:55:25.955305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 139/283] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:25 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:26.218669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:26.231626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-347c3d18-fbd1-4be4-9a54-0fa50b80a287 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.598247 Sep 11 00:55:26.239640 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 145/284] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:25 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:26.992660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:26.992660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:27.010069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:27.010776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:27.011456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:27.012342 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:27.012961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:27.013550 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:27.267658 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:27.268285 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:27.282905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:27.287447 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:27.287905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:27.288731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:27.289164 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:27.289593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:27.703303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:27.713992 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c33a8f0b-45a0-40a2-bf38-4d7d9725b676 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.450525 Sep 11 00:55:27.718011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 146/285] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:27 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:27.817518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:27.831620 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:27.840568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:27.846441 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fd3e0e0b-f25c-44d0-b998-0cd1198ae9ad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1738 microversion: 2.20 time: 0.862302 Sep 11 00:55:27.849844 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 140/286] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:26 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1738 bytes in 867 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:28.118027 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c89708be-c1a6-4d3f-a4bc-9148e7a9b35e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Action: 'create', calling method: .version_select at 0x7fc01ade93a0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-975872804", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGVo70ryRagGd0T9Lza/WLt8yUmSufdPIE92ah0uUGEy5KWtqElolM/7dEoQ4AUjWKsvSqzpAaUTBxdbQJRyhoqNnKsWIm4ubAIYfSfeeq9bgvkZbNaQosWTuU4T0iqAMg=="}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:28.139648 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-c89708be-c1a6-4d3f-a4bc-9148e7a9b35e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for project a1368870ddbd49be857e9ac128bc684e. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:55:28.144168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-c89708be-c1a6-4d3f-a4bc-9148e7a9b35e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for user e1012f856881485cb40d916131a5d763 and project a1368870ddbd49be857e9ac128bc684e. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:55:28.223753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c89708be-c1a6-4d3f-a4bc-9148e7a9b35e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.340503 Sep 11 00:55:28.232697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 147/287] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:55:27 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:28.411114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-e38c3c33-5b16-44f4-a412-b5faba4eaeec req-77347f52-e718-46aa-9b2f-99dc30c81020 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a843e04-0782-4e1e-b379-9be9a7106442", "tag": "d120b4b9-10bc-4478-8187-9b7014b08cef"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:28.421426 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e38c3c33-5b16-44f4-a412-b5faba4eaeec req-77347f52-e718-46aa-9b2f-99dc30c81020 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:28.421674 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e38c3c33-5b16-44f4-a412-b5faba4eaeec req-77347f52-e718-46aa-9b2f-99dc30c81020 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:28.422044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e38c3c33-5b16-44f4-a412-b5faba4eaeec req-77347f52-e718-46aa-9b2f-99dc30c81020 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:28.466602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-e38c3c33-5b16-44f4-a412-b5faba4eaeec req-77347f52-e718-46aa-9b2f-99dc30c81020 service nova] Creating event network-changed:d120b4b9-10bc-4478-8187-9b7014b08cef for instance 9a843e04-0782-4e1e-b379-9be9a7106442 on hv-cinder-maste Sep 11 00:55:28.486151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-e38c3c33-5b16-44f4-a412-b5faba4eaeec req-77347f52-e718-46aa-9b2f-99dc30c81020 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078692 Sep 11 00:55:28.487501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 141/288] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:28.493794 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f721beed-61a7-4f73-83d0-cb36c709b30d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Action: 'create', calling method: .version_select at 0x7fc01ad4d1f0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-371702567", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLBXi1RLi7QtGLleAiveboLCKxBne4G+mZiyHAMaq2rEeFoFAHR9lJP4dGriguEUBuErYyrDkUA/4vAqJkFlPs+WF6eDiEtPWXeU0jTtga5CI6Uyy7wsKkCeCPUReoZWBQ=="}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:28.571284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-f721beed-61a7-4f73-83d0-cb36c709b30d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Getting quotas for project 299fcf2ee8b24a4fbb27ed83954c8e24. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:55:28.580138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-f721beed-61a7-4f73-83d0-cb36c709b30d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Getting quotas for user 953d7ed2081c48c79a1a5660370b861b and project 299fcf2ee8b24a4fbb27ed83954c8e24. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:55:28.638896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f721beed-61a7-4f73-83d0-cb36c709b30d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.151005 Sep 11 00:55:28.641630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 148/289] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:55:28 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:28.742484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:28.743408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:28.761591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:28.763346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:28.766159 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:28.767924 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:28.769682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:28.771660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:28.787818 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:28.808599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-459419004", "imageRef": "91518f1d-b451-4058-9315-f1fba36dce2e", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-371702567", "networks": [{"uuid": "9fa144fa-5fc9-4425-a84d-a5df211c8436"}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:28.882334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] validate_networks() for [('9fa144fa-5fc9-4425-a84d-a5df211c8436', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:55:29.111600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:29.125057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b57668f3-ddc8-422c-8ee0-9bcdd3c6599a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.387469 Sep 11 00:55:29.127578 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 142/290] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:28 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:29.151065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:29.151682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:29.158590 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:29.159378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:29.159817 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:29.160533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:29.160988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:29.161414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:29.574999 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:29.583616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:29.585080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:29.586314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-13033f04-0871-42b3-9012-5de0792e26c2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1720 microversion: 2.20 time: 0.439394 Sep 11 00:55:29.586899 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 143/291] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:29 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1720 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:29.607747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-fd427c29-1246-4214-93e2-2427c9d02cae req-f0563e31-39f1-4e0e-aa19-cbd7963fd3f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "name": "network-vif-plugged", "status": "completed", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:29.639547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-fd427c29-1246-4214-93e2-2427c9d02cae req-f0563e31-39f1-4e0e-aa19-cbd7963fd3f5 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:29.640105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-fd427c29-1246-4214-93e2-2427c9d02cae req-f0563e31-39f1-4e0e-aa19-cbd7963fd3f5 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:29.642232 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-fd427c29-1246-4214-93e2-2427c9d02cae req-f0563e31-39f1-4e0e-aa19-cbd7963fd3f5 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:29.669735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-fd427c29-1246-4214-93e2-2427c9d02cae req-f0563e31-39f1-4e0e-aa19-cbd7963fd3f5 service nova] Creating event network-vif-plugged:549957cc-ef50-4e12-9608-80b215c4bfb4 for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c on hv-cinder-maste Sep 11 00:55:29.674843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:55:29.676520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:55:29.677176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:55:29.677752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:55:29.678407 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:55:29.684430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-fd427c29-1246-4214-93e2-2427c9d02cae req-f0563e31-39f1-4e0e-aa19-cbd7963fd3f5 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081605 Sep 11 00:55:29.692372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 144/292] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:29.695781 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:29.697909 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:29.698530 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:29.704011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:29.704763 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:29.705361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:29.755269 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Getting quotas for project 299fcf2ee8b24a4fbb27ed83954c8e24. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:55:29.760492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Getting quotas for user 953d7ed2081c48c79a1a5660370b861b and project 299fcf2ee8b24a4fbb27ed83954c8e24. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:55:29.773284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:55:29.776601 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='91518f1d-b451-4058-9315-f1fba36dce2e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:55:29.908424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d047ec73-eba9-4c87-8850-eb1a943602c5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.100329 Sep 11 00:55:29.919421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 149/293] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:55:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 1116 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:55:29.933904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-12fa4b21-6e51-4c7d-886b-9e1143138d8f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:29.935222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-12fa4b21-6e51-4c7d-886b-9e1143138d8f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:30.157370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:30.157992 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:30.178370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:30.178954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:30.179415 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:30.180246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:30.180690 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:30.192358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:30.246042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-12fa4b21-6e51-4c7d-886b-9e1143138d8f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:30.253379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-12fa4b21-6e51-4c7d-886b-9e1143138d8f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1257 microversion: 2.1 time: 0.332315 Sep 11 00:55:30.256339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 145/294] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:29 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1257 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:30.491588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:30.501723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c494bf6b-cac5-43ec-bf73-93dfb5556ee0 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.351529 Sep 11 00:55:30.502965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 150/295] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:30 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:30.615756 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:30.616408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:30.624515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:30.625021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:30.625725 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:30.626442 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:30.626962 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:30.627395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:30.790516 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:30.798891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:30.800334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:30.801503 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7cd64275-a5d1-49f0-828e-74fccb8014ba tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1720 microversion: 2.20 time: 0.189699 Sep 11 00:55:30.802483 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 146/296] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:30 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1720 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:31.282732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:31.283366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:31.290124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:31.290622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:31.291138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:31.291834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:31.292288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:31.292708 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:31.359856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:31.370886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b41e8819-84fa-4a63-96d6-eba8d0abba89 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1261 microversion: 2.1 time: 0.092031 Sep 11 00:55:31.371467 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 151/297] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:31 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1261 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:31.527318 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:31.527984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:31.540184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:31.540681 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:31.541120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:31.541836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:31.542288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:31.542711 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:31.721330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:31.734069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ff2fd48f-b8ab-4efd-bb09-911e3ae1e319 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.209363 Sep 11 00:55:31.738988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 147/298] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:31 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:31.833638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:31.834496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:31.841546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:31.842309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:31.842984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:31.843883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:31.844340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:31.844948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:31.986608 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:31.995738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:31.997203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:31.998379 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4f0fc757-af68-4fda-bddf-19b844c44e16 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1720 microversion: 2.20 time: 0.170207 Sep 11 00:55:31.998972 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 152/299] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:31 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1720 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:32.395285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:32.395568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:32.402470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:32.403021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:32.403482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:32.404173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:32.404637 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:32.405057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:32.472598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:32.486025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-88ab5ac9-c6e2-47dd-9eb2-4c55d299780a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1253 microversion: 2.1 time: 0.094921 Sep 11 00:55:32.486706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 148/300] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:32 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1253 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:32.757903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:32.758521 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:32.765544 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:32.766026 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:32.766460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:32.767172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:32.767627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:32.768052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:32.914655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:32.922449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1875c51e-16ed-436c-bcd3-55794647b3cb tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1261 microversion: 2.1 time: 0.169128 Sep 11 00:55:32.923050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 153/301] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:32 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1261 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:33.025053 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:33.025593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:33.032211 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.032771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.033208 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.033898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.034662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.035116 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.185886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-f4651c68-c89d-4888-a874-e8d1153b24e0 req-5eb3ca07-6eb4-4203-9922-c821feb75a61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "name": "network-vif-unplugged", "status": "completed", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:33.195983 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f4651c68-c89d-4888-a874-e8d1153b24e0 req-5eb3ca07-6eb4-4203-9922-c821feb75a61 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.196596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f4651c68-c89d-4888-a874-e8d1153b24e0 req-5eb3ca07-6eb4-4203-9922-c821feb75a61 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.197228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f4651c68-c89d-4888-a874-e8d1153b24e0 req-5eb3ca07-6eb4-4203-9922-c821feb75a61 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.209294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-f4651c68-c89d-4888-a874-e8d1153b24e0 req-5eb3ca07-6eb4-4203-9922-c821feb75a61 service nova] Creating event network-vif-unplugged:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:55:33.215834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-f4651c68-c89d-4888-a874-e8d1153b24e0 req-5eb3ca07-6eb4-4203-9922-c821feb75a61 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033609 Sep 11 00:55:33.216940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 154/302] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:55:33.223967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:33.235803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:33.237412 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:33.238587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-831c6678-063e-4a70-b488-70c28fe49f67 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1723 microversion: 2.20 time: 0.217639 Sep 11 00:55:33.239333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 149/303] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:33 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1723 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:33.263811 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3dc941c9-df60-4826-90b5-50998786adf4"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:33.266283 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:33.273730 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.274430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.274910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.275875 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.276417 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.276853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.508710 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:33.509319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:33.515214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.515697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.516129 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.516817 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.517273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.517692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.578555 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-31aae26e-7bd9-4da0-abe9-36d6041dd462 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.317957 Sep 11 00:55:33.579227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 155/304] 10.222.0.27 () {62 vars in 1364 bytes} [Sun Sep 11 00:55:33 2022] POST /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments => generated 188 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:55:33.589049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:33.602041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c5da62f7-1483-4d79-86e4-97fe06f5f9d5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1309 microversion: 2.1 time: 0.097456 Sep 11 00:55:33.602884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 150/305] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:33 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1309 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:33.712841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:33.714229 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:33.722773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.723564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.724162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.725051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.725479 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.725944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.774497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=126979) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 11 00:55:33.844825 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e8e56750-ead9-48a4-969f-1438924e9d97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/action" status: 202 len: 0 microversion: 2.20 time: 0.137431 Sep 11 00:55:33.845431 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 156/306] 10.222.0.27 () {62 vars in 1319 bytes} [Sun Sep 11 00:55:33 2022] POST /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/action => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:55:33.854316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:33.855172 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:33.862060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.862529 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.863179 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.864105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.864747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.865323 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.949473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:33.950102 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:33.953189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=5,nova_api:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:33.957339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.957826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.958460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:33.959225 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:33.959681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:33.960330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:34.057793 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:34.068869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:34.071593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:34.072818 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-64a1615c-31c8-49e9-8149-4ca8738bfce9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1809 microversion: 2.20 time: 0.223207 Sep 11 00:55:34.073363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 151/307] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:33 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1809 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:34.161245 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:34.172420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7b2891dc-3464-411f-9051-da97d8fa8c6a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1444 microversion: 2.1 time: 0.226810 Sep 11 00:55:34.172991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 157/308] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:33 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1444 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:34.197255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:34.198363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:34.206114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:34.206609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:34.207069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:34.207765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:34.208220 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:34.208781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:34.384198 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:34.396700 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d2300624-38db-4bae-a4f0-b470bf5d406a tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1444 microversion: 2.1 time: 0.204792 Sep 11 00:55:34.398104 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 152/309] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:34 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1444 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:34.624652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:34.625694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:34.635332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:34.637819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:34.639483 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:34.640673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:34.641342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:34.641789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:34.756091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:34.768045 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0ca6402b-74dd-478e-8dbe-bf6a359ea060 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1315 microversion: 2.1 time: 0.147521 Sep 11 00:55:34.768678 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 158/310] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:34 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1315 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:35.098953 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:35.100152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:35.107749 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:35.108469 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:35.109056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:35.110529 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:35.111178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:35.111752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:35.323952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:35.336760 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:35.338301 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:35.339651 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a55aea36-5e9d-479f-8af6-dd2ec7fb08aa tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1809 microversion: 2.20 time: 0.244606 Sep 11 00:55:35.342832 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 153/311] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:35 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1809 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:35.792392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:35.793675 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:35.800083 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:35.811596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:35.812344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:35.813262 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:35.813912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:35.814516 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:35.935751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:35.946267 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d8c5737f-6784-4681-9ea6-b77ecc30595b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1315 microversion: 2.1 time: 0.158238 Sep 11 00:55:35.947136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 159/312] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:35 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:36.365307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:36.365852 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:36.383729 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:36.384563 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:36.385344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:36.386562 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:36.387625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:36.396060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:36.632072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:36.641918 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:36.644151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:36.645706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c0632e47-4db8-4ecd-bda7-f7cb830b8fc3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.284487 Sep 11 00:55:36.646457 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 154/313] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:36 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:36.974870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:36.976511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:36.983600 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:36.984144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:36.984590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:36.985337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:36.985820 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:36.986462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:37.188902 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:37.192224 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:37.204288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4e3fe4c2-a332-4075-b55d-2a32e0a14e9e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.233105 Sep 11 00:55:37.206135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 160/314] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:36 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:37.675848 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:37.676491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:37.683654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:37.684137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:37.684575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:37.685265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:37.685715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:37.686136 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:37.828667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-ad8fd4e8-5ac1-458a-8def-a3f5c1dd4594 req-aa54b495-a3f0-44d1-978e-11aa42e3c5a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:37.854829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-ad8fd4e8-5ac1-458a-8def-a3f5c1dd4594 req-aa54b495-a3f0-44d1-978e-11aa42e3c5a6 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:37.855637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-ad8fd4e8-5ac1-458a-8def-a3f5c1dd4594 req-aa54b495-a3f0-44d1-978e-11aa42e3c5a6 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:37.856114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-ad8fd4e8-5ac1-458a-8def-a3f5c1dd4594 req-aa54b495-a3f0-44d1-978e-11aa42e3c5a6 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:37.898081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-ad8fd4e8-5ac1-458a-8def-a3f5c1dd4594 req-aa54b495-a3f0-44d1-978e-11aa42e3c5a6 service nova] Creating event network-changed:549957cc-ef50-4e12-9608-80b215c4bfb4 for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c on hv-cinder-maste Sep 11 00:55:37.905718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-ad8fd4e8-5ac1-458a-8def-a3f5c1dd4594 req-aa54b495-a3f0-44d1-978e-11aa42e3c5a6 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082056 Sep 11 00:55:37.909309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 161/315] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:37.936189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:37.973108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:37.975873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:37.978464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3c417b58-10d3-4450-ac1d-fd1027e69ecc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.306496 Sep 11 00:55:37.979576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 155/316] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:37 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:38.230358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:38.230998 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:38.237188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:38.237673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:38.238105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:38.238807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:38.239274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:38.239696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:38.409698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:38.424214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0e9dc2b2-6867-4596-a374-3e1c4002d705 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.199465 Sep 11 00:55:38.425403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 162/317] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:38 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:39.012482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:39.014144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:39.021613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:39.024437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:39.025146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:39.026308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:39.033472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:39.035787 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:39.218737 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:39.227755 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:39.229200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:39.230367 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-96e362aa-0d57-4a10-9ff8-fb8b6916ec18 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.228351 Sep 11 00:55:39.230959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 156/318] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:39 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:39.448037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:39.448627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:39.454919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:39.455400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:39.455828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:39.456506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:39.456952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:39.457369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:39.601949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:39.618280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d93c3fe7-d3c5-4ede-9c9c-0a8e0c117a95 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.174325 Sep 11 00:55:39.635955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 163/319] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:39 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:40.258243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:40.258887 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:40.261474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:40.266911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:40.267404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:40.267838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:40.268539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:40.268995 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:40.269429 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:40.407900 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:40.417280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:40.418992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:40.420148 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-96f9e4b0-49a1-4e2e-beab-c857165f385b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.166563 Sep 11 00:55:40.420703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 157/320] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:40 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:40.482863 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2f9be68f-b939-4124-aeea-e4b84fd8b4f7"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:40.485202 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:40.487201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:40.488070 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=14 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:40.494750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:40.495256 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:40.495695 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:40.496386 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:40.496837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:40.497282 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:40.645108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:40.646411 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:40.655800 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:40.656651 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:40.657449 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:40.658603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:40.659585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:40.660092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:40.818272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:40.840713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7e2ef942-a61f-4068-8108-1b8cbb19d54f tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.202453 Sep 11 00:55:40.841689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 158/321] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:40 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:41.454296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:41.454980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:41.461552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:41.462050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:41.462491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:41.463233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:41.463691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:41.464121 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:41.637958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:41.649221 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:41.650715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:41.652122 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7f332f50-7b72-4a3e-9bb3-0cacc5431a97 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.201873 Sep 11 00:55:41.652875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 159/322] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:41 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:41.868809 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:41.869461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:41.875728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:41.876233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:41.876668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:41.877382 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:41.877835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:41.878259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:41.901754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:42.017273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:42.025948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a4371e70-4d9a-48c5-836a-8142e4ecd799 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.161763 Sep 11 00:55:42.026430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 160/323] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:41 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:42.677458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:42.678093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:42.684533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:42.685025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:42.685460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:42.686241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:42.686618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:42.687101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:42.826388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:42.837753 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:42.839568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:42.841048 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3aa81d4d-e1da-4ebc-b18e-0fbc0a921ef8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.167475 Sep 11 00:55:42.841864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 161/324] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:42 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:43.050934 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:43.051553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:43.058179 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:43.058665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:43.059148 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:43.059839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:43.060307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:43.060737 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:43.193471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:43.201500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7be4c270-5ba5-4604-9bf1-d6a30da8b191 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.154791 Sep 11 00:55:43.202065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 162/325] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:43 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:43.867745 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:43.868462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:43.876127 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:43.876587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:43.877021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:43.877806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:43.878375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:43.878707 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:44.033017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:44.043491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:44.044945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:44.046090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9ce00dc0-f756-4966-a8a5-cf15b36fe1ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.183269 Sep 11 00:55:44.046639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 163/326] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:43 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:44.224152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:44.224761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:44.231067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:44.231546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:44.231976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:44.232660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:44.233108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:44.233524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:44.371919 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:44.380552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0fd65f82-3d58-4b5c-a666-2ece75a9eeab tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.160113 Sep 11 00:55:44.381308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 164/327] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:44 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:45.071421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:45.072104 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:45.078545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:45.079063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:45.079495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:45.080180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:45.080630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:45.081115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:45.207394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:45.217912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:45.219372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:45.220518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4be4a9cb-8fb7-414c-bbf1-00b06c6aba93 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.153106 Sep 11 00:55:45.221068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 165/328] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:45 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:45.411196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:45.412184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:45.419453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:45.419947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:45.420380 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:45.421086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:45.421566 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:45.421993 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:45.576168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:45.585233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9174e225-7081-4470-8f2e-27ac1a5a4a51 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.183817 Sep 11 00:55:45.586042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 166/329] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:45 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:46.250504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:46.251226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:46.258114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:46.258582 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:46.259043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:46.259727 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:46.260178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:46.260623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:46.394036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:46.403591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:46.405037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:46.406191 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-01dab693-cd2d-4ec2-acc0-2732b4c881e2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.159571 Sep 11 00:55:46.406750 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 167/330] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:46 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:55:46.613789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:46.614310 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:46.620414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:46.620893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:46.621321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:46.622117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:46.622858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:46.623511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:46.798331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:46.807621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ee9ad303-2987-4647-bb62-fe213d46b9db tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.197745 Sep 11 00:55:46.808427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 168/331] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:46 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:47.432402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:47.433116 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:47.439731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:47.440294 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:47.440795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:47.441615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:47.442243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:47.442758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:47.590470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:47.599108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:47.600575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:47.602280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ed82861a-e2fe-4d51-a90e-34b553c13d75 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.174157 Sep 11 00:55:47.603073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 169/332] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:47 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:47.831385 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:47.831995 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:47.843009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:47.843009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:47.843009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:47.843009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:47.843568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:47.843568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:48.031928 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:48.040872 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7b65fd30-fd08-4d95-a07e-70125032db26 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.212754 Sep 11 00:55:48.041442 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 170/333] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:47 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:48.346697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-285327847", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-975872804", "security_groups": [{"name": "tempest-secgroup-smoke-926362896"}], "block_device_mapping_v2": [{"uuid": "570647bd-2013-4675-8b3d-96669589b642", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "6df3e98e-548f-496e-bd19-e33fc45eb877"}]}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:48.458060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] validate_networks() for [('6df3e98e-548f-496e-bd19-e33fc45eb877', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:55:48.818182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:55:48.818608 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:55:48.819138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:55:48.819537 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:55:48.820658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:55:48.829346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:48.829829 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:48.830265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:48.831027 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:48.831488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:48.831915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:48.848090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for project a1368870ddbd49be857e9ac128bc684e. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:55:48.852189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for user e1012f856881485cb40d916131a5d763 and project a1368870ddbd49be857e9ac128bc684e. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:55:48.907689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:55:48.910052 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='570647bd-2013-4675-8b3d-96669589b642',volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:55:49.040375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d836efc0-ff99-49de-b6aa-486b22f2b137 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.697486 Sep 11 00:55:49.041187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 171/334] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:55:48 2022] POST /compute/v2.1/servers => generated 405 bytes in 699 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 11 00:55:49.045760 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:49.046371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:49.052806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:49.053299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:49.053738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:49.054990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:49.055443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:49.056446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:49.194137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:49.203931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:49.205455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:49.206641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ea971bb9-c72e-491a-804a-deab684afb5e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.164628 Sep 11 00:55:49.207240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 172/335] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:49 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:49.212619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3501b2f8-1c38-406a-b0dc-b2ec0afed7b8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:49.213222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3501b2f8-1c38-406a-b0dc-b2ec0afed7b8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:49.272401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3501b2f8-1c38-406a-b0dc-b2ec0afed7b8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:49.283748 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3501b2f8-1c38-406a-b0dc-b2ec0afed7b8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1093 microversion: 2.1 time: 0.075462 Sep 11 00:55:49.284796 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 173/336] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:49 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1093 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:49.290392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:49.291248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:49.293228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:49.302656 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:49.303331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:49.303959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:49.304891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:49.305519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:49.306103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:49.616361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cc630f58-e475-41c3-86d5-f391e24a4da9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 9.137451 Sep 11 00:55:49.623384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 164/337] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:55:40 2022] POST /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments => generated 194 bytes in 9145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:49.778679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:49.786688 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-96008185-2d4c-4fb4-aedb-6a81b9b693e5 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.500640 Sep 11 00:55:49.787281 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 174/338] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:49 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:50.058496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-7af477ee-2fad-4a6f-bd76-f8af6d4a5015 req-c697194f-cfbf-44d9-b4df-b8243028af58 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:50.089834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-7af477ee-2fad-4a6f-bd76-f8af6d4a5015 req-c697194f-cfbf-44d9-b4df-b8243028af58 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.091679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-7af477ee-2fad-4a6f-bd76-f8af6d4a5015 req-c697194f-cfbf-44d9-b4df-b8243028af58 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.093200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-7af477ee-2fad-4a6f-bd76-f8af6d4a5015 req-c697194f-cfbf-44d9-b4df-b8243028af58 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.107102 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-7af477ee-2fad-4a6f-bd76-f8af6d4a5015 req-c697194f-cfbf-44d9-b4df-b8243028af58 service nova] Creating event network-changed:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:55:50.117438 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-7af477ee-2fad-4a6f-bd76-f8af6d4a5015 req-c697194f-cfbf-44d9-b4df-b8243028af58 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084377 Sep 11 00:55:50.119371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 165/339] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:50.232757 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:50.233359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:50.239494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.239969 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.240398 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.241078 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.241577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.242002 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.312077 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:50.312652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:50.319045 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.319557 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.320001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.321189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.321652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.322083 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.436503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:50.455024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-78b99454-421c-4f4f-90d1-019ce23d6025 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1135 microversion: 2.1 time: 0.146619 Sep 11 00:55:50.455749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 166/340] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:50 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1135 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:50.540195 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:50.550017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:50.551563 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:50.552908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d767c701-12e7-4af2-ba9f-7f51f1308826 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.324037 Sep 11 00:55:50.553766 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 175/341] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:50 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:50.816718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:50.817415 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:50.829127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.829633 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.830076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.830814 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:50.831386 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:50.832089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:50.977243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:50.985278 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6f46ec51-482b-4955-bddf-42470e3dd6d2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.180252 Sep 11 00:55:50.985848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 167/342] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:50 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:51.479074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:51.479740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:51.486346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:51.486857 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:51.487296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:51.487986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:51.488443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:51.488894 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:51.550680 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:51.559184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fad9feb5-cc8d-436b-b7c0-31ecad8465b2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1135 microversion: 2.1 time: 0.084787 Sep 11 00:55:51.559745 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 176/343] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:51 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1135 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:51.581419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:51.581947 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:51.588102 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:51.588590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:51.589028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:51.589717 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:51.590175 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:51.590602 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:51.723589 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:51.732830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:51.734485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:51.735667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7b2fc428-64a0-4f93-874d-cf8e068c88d9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1751 microversion: 2.20 time: 0.158211 Sep 11 00:55:51.736227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 168/344] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:51 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1751 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:52.008140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:52.008747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:52.014715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:52.015231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:52.015667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:52.016355 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:52.016813 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:52.017237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:52.149544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:52.157072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-766a0ad8-f193-49ba-913d-da646c8acb80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.152932 Sep 11 00:55:52.157634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 177/345] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:52 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:52.581570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:52.582183 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:52.589408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:52.589903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:52.590346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:52.591077 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:52.591540 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:52.591993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:52.653004 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:52.660837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-22a8908f-a817-46d9-9290-e3be2c0993ba tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1135 microversion: 2.1 time: 0.083221 Sep 11 00:55:52.661394 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 169/346] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:52 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1135 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:52.761820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:52.762438 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:52.769774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:52.770269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:52.770731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:52.771932 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:52.772398 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:52.773377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:52.908758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:52.912486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:52.918465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:52.919961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:52.921112 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4ab2cd8d-4187-43f1-a508-7942f0093aa2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.163280 Sep 11 00:55:52.921672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 178/347] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:52 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:53.191927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:53.192683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:53.208323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:53.209915 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:53.210719 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:53.214592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:53.215300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:53.215861 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:53.383330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:53.391906 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d1c48fce-aad6-45be-a182-1edee17c0972 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.215696 Sep 11 00:55:53.392476 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 170/348] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:53 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:53.685219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:53.685747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:53.691713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:53.692913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:53.692913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:53.693453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:53.693739 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:53.694215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:53.761091 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:53.771016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-eb4c7523-fbf3-4bcf-a1de-81eb36eb3ff1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1135 microversion: 2.1 time: 0.089906 Sep 11 00:55:53.772293 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 179/349] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:53 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1135 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:53.946022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:53.946633 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:53.954051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:53.954424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:53.954889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:53.955591 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:53.956069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:53.956475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.038293 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:54.040827 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:54.048114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.048771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.049209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.049911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.050535 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.051057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.076786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lazy-loading 'flavor' on Instance uuid eb3cc25c-09a9-47c9-bc2e-d5227478ab70 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:55:54.182618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:54.195405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:54.198499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:54.200341 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-335a1e19-1fe0-47b0-9f44-5aae63584a34 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.258425 Sep 11 00:55:54.201269 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 171/350] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:53 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:54.220151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0390f346-c4da-47a0-83ad-bb7c29ef8d29 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/action" status: 200 len: 32 microversion: 2.1 time: 0.186357 Sep 11 00:55:54.220779 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 180/351] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:55:54 2022] POST /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/action => generated 32 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 00:55:54.231175 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:54.231515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:54.238510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.239065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.239728 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.240758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.243005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.243617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.407788 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:54.416106 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:54.416273 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9554cbba-d79f-4798-b3f5-c15e231db740 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.190006 Sep 11 00:55:54.416815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:54.416962 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 172/352] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:54 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:54.423138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.423625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.424062 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.424754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.425208 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.425629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.558952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:54.566994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-38cde498-db6c-417f-8be4-62787cfbe019 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.155117 Sep 11 00:55:54.567759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 181/353] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:54 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:54.796116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:54.796807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:54.805830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.806899 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.807414 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.808254 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:54.809253 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:54.809757 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:54.891393 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:54.899563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6b646edc-ea5b-4b66-bd6e-9e3aa14b1ca4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1209 microversion: 2.1 time: 0.107800 Sep 11 00:55:54.900329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 173/354] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:54 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1209 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:55.225961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:55.234668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:55.243576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.244074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.244511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.247494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.247968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.248409 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.440953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:55.442756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:55.454539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.458253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:55.459310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.462143 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.462934 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.464071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.467080 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.479352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:55.485717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:55.487018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-33672164-32af-45d7-9680-528f628f27b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.265277 Sep 11 00:55:55.515253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 182/355] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:55 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:55.600588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:55.602305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:55.616488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.618934 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.619883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.621337 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.623075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.624106 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.812699 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:55.838019 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-50d341d6-c153-4882-9f6a-0f8f7b315185 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.401135 Sep 11 00:55:55.840747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 174/356] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:55 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:55.925551 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:55.926617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:55.941639 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.945453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.947116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:55.949503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:55.951021 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:55.953452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:56.000747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:56.014551 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e7122922-c84f-4c53-8ffb-bddd84ea8746 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.421991 Sep 11 00:55:56.015570 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 183/357] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:55 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:56.230622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:56.240404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-edce139c-cb2d-4908-9b84-0953dc34af64 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1276 microversion: 2.1 time: 0.319999 Sep 11 00:55:56.245098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 175/358] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:55 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1276 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:56.523605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:56.524224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:56.531709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:56.532214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:56.532658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:56.533375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:56.534099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:56.534690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:56.690260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:56.700269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:56.701716 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:56.702908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f09f7e92-cdfa-4d30-a37a-e95383a29de8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.184484 Sep 11 00:55:56.703474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 184/359] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:56 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:56.864633 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:56.865260 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:56.871825 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:56.872323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:56.872760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:56.873462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:56.873914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:56.874338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.016041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:57.026350 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2b5ad9f2-a7c7-4237-ad97-d8bba53033cc tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.165728 Sep 11 00:55:57.026955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 176/360] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:56 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:57.058952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:57.059631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:57.065981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.066472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.066960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.067658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.068107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.068527 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.214067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:57.221842 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-70caab8a-0015-45fa-ba63-319a0b120653 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.168029 Sep 11 00:55:57.222612 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 185/361] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:57 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:57.270626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:57.271791 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:57.280526 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.281020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.281462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.282204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.282904 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.283145 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.307576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:57.361069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-b8c2c9cd-9ce5-44f1-821e-be4aea472ca6 req-35acd8f5-711e-477f-b9d6-a76838bd1dd0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "name": "network-vif-plugged", "status": "completed", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:55:57.386913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-b8c2c9cd-9ce5-44f1-821e-be4aea472ca6 req-35acd8f5-711e-477f-b9d6-a76838bd1dd0 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.387883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-b8c2c9cd-9ce5-44f1-821e-be4aea472ca6 req-35acd8f5-711e-477f-b9d6-a76838bd1dd0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.388609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-b8c2c9cd-9ce5-44f1-821e-be4aea472ca6 req-35acd8f5-711e-477f-b9d6-a76838bd1dd0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.403023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-b8c2c9cd-9ce5-44f1-821e-be4aea472ca6 req-35acd8f5-711e-477f-b9d6-a76838bd1dd0 service nova] Creating event network-vif-plugged:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:55:57.408366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-b8c2c9cd-9ce5-44f1-821e-be4aea472ca6 req-35acd8f5-711e-477f-b9d6-a76838bd1dd0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050928 Sep 11 00:55:57.409877 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 186/362] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:55:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:55:57.485361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:57.493659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fd51ae02-4080-4aff-9710-ca1e6670c239 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.228894 Sep 11 00:55:57.494822 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 177/363] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:57 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:57.730331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:57.731195 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:57.738115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.738529 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.738992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.739700 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:57.740158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:57.740583 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:57.885548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:57.895098 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:57.896572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:57.897783 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-33fa7669-dcbe-4333-ad61-259b3827667f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.172535 Sep 11 00:55:57.898970 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 187/364] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:57 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:58.050535 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:58.051185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:58.058292 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.058834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.059279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.059973 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.060451 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.060862 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.234951 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:58.244336 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-42c265b6-9679-442d-8313-6dd619ab84ac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.197518 Sep 11 00:55:58.245157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 178/365] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:58 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:58.251654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:58.252282 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:58.258698 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.259218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.259660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.260403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.260860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.261287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.404578 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:58.413724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fe25bae8-5ed3-4e48-8698-5e155b018801 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.171630 Sep 11 00:55:58.414371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 188/366] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:58 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:58.517848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:58.518507 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:58.526656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.527577 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.528330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.529306 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.529765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.530193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.676844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:58.685522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3edeb287-8e23-4444-b4ce-1237110a9282 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.171582 Sep 11 00:55:58.686288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 179/367] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:58 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:58.925419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:58.926004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:58.928407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:58.934340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.934866 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.935348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:58.936060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:58.936515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:58.936938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.075697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:59.084577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:59.086577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:59.087795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-51fdbbb1-d2de-45d0-9397-bef76241993a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.166421 Sep 11 00:55:59.088537 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 189/368] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:55:58 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:55:59.268106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:59.268815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:59.275412 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:59.275906 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:59.276346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.277050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:59.277515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:59.277943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.413519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:59.422238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1601c2c1-a5d9-4a99-9843-8596b67dc98c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.157969 Sep 11 00:55:59.430072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 180/369] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:59 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:55:59.441373 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:59.442004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:59.444115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=8 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:59.451010 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:59.451585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:59.452168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.452973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:59.453498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:59.454033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.599594 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:59.609234 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a3692c21-011e-481b-b00b-72c4ea5e1b37 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.175549 Sep 11 00:55:59.609813 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 190/370] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:59 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:55:59.708582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:55:59.709199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:55:59.716062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:59.716558 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:59.717031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.717731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:55:59.718193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:55:59.718624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:55:59.856084 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:55:59.864062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1de00ecd-de5e-44c5-a07c-9ebd69f8cc5d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.159320 Sep 11 00:55:59.866199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 181/371] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:55:59 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:00.113505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:00.114278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:00.121377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.121874 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.122312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.123051 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.123514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.123945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.293089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:00.302078 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:00.303467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:00.305193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-22b207ca-79bd-4af8-baee-837aacda98fc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.195668 Sep 11 00:56:00.306008 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 191/372] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:00 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:00.447279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:00.447904 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:00.454715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.455262 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.455701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.456469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.456947 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.457376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.594500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:00.603592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fb0b9dce-0bb1-4257-8c83-f21f784c788f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.160678 Sep 11 00:56:00.604160 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 182/373] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:00 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:00.634299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:00.634939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:00.641204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.641688 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.642114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.642821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.643270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.643693 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.781018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:00.790100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e3f0198b-b552-438d-b3bf-db0b09c90035 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.160082 Sep 11 00:56:00.791041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 192/374] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:00 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:00.889972 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:00.890501 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:00.897374 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.897928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.898360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:00.899097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:00.899554 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:00.899978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.040064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:01.048376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a357eadb-7277-4005-aaa0-0997a416d4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.162618 Sep 11 00:56:01.049187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 183/375] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:00 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:01.332027 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:01.332681 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:01.339348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:01.339836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:01.340269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.340965 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:01.341413 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:01.341882 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.477506 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:01.486366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:01.487834 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:01.489058 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5f885475-a3e3-4fd6-b25e-e83c43d22cf1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.161396 Sep 11 00:56:01.490256 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 193/376] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:01 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:01.627654 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:01.628338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:01.635188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:01.635673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:01.636102 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.636797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:01.637235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:01.637659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.779726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:01.790338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-40a139da-6696-4138-92d7-32f0e65b27a6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.166278 Sep 11 00:56:01.790898 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 184/377] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:01 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:01.813522 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:01.814482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:01.821942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:01.822438 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:01.822902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.823609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:01.824249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:01.824819 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:01.829226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:01.969714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:01.977779 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3c95f842-0909-4715-bfd6-8f1aa493137a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.168156 Sep 11 00:56:01.978541 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 194/378] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:01 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:02.072321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:02.072937 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:02.081705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:02.081705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:02.081705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:02.081705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:02.082141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:02.082141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:02.229730 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:02.237471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-98352016-4ec6-4d21-8bdc-055a5affdaf2 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.169575 Sep 11 00:56:02.246834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 185/379] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:02 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:02.514766 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:02.515830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:02.522309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:02.523329 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:02.523762 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:02.524466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:02.524921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:02.525374 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:02.660950 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:02.670418 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:02.671888 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:02.673049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-81727618-6bf6-484f-b465-8a95c8e346a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.162539 Sep 11 00:56:02.673643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 195/380] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:02 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:02.814643 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:02.815434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:02.823208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:02.823699 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:02.824128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:02.824844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:02.825291 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:02.825709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:02.964289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:02.972608 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e326ea64-2013-4473-81f5-66cbafafc7d6 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.162879 Sep 11 00:56:02.973172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 186/381] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:02 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:03.002328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:03.003037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:03.010611 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:03.011158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:03.011714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:03.012541 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:03.013065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:03.013488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:03.169666 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:03.177560 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-68a585d9-a48c-4095-8665-e079bdb387f3 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.179757 Sep 11 00:56:03.178134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 196/382] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:02 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:03.260516 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:03.261166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:03.267152 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:03.267687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:03.268079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:03.268773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:03.269228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:03.269657 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:03.404764 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:03.414679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d89bf4cb-23e4-4c44-bbb1-b227f215107f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.158574 Sep 11 00:56:03.415349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 187/383] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:03 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:03.704253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:03.704876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:03.712435 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:03.712968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:03.713432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:03.714160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:03.714643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:03.715182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:03.868108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:03.877236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:03.878840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:03.880141 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6d59b5d2-1825-4b2e-b48e-868ff1aaab87 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.179935 Sep 11 00:56:03.880761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 197/384] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:03 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:03.998309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:03.999690 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:04.002012 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=61 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:04.008485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.008981 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.009412 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.010115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.010572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.011024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.155632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:04.164208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-adfd08fb-eb6e-4073-9967-f44814a67e8f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.169760 Sep 11 00:56:04.165099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 188/385] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:03 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:04.202022 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:04.202641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:04.208774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.209254 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.209682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.211373 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.211858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.212285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.348607 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:04.356787 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e2042492-973d-4ef0-9227-e2813205fb24 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.158751 Sep 11 00:56:04.357554 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 198/386] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:04 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:04.437173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:04.437786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:04.443798 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.444988 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.445435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.446119 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.446572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.447028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.579618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:04.587632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-df01574f-cab1-4a93-9f16-f16931d6cf93 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.154320 Sep 11 00:56:04.588430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 189/387] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:04 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:04.907299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:04.907851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:04.915156 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.915648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.916083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:04.916776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:04.917235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:04.917660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.059524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:05.068726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:05.070273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:05.071468 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5de6970e-a909-4e0a-b103-42f028dae805 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.168188 Sep 11 00:56:05.072047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 199/388] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:04 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:05.187733 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:05.188290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:05.194712 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.195237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.195682 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.196384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.196852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.197283 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.329768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:05.338500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6289c9ac-5a11-4b6e-824c-8680dc04275a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.154673 Sep 11 00:56:05.339301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 190/389] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:05 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:05.379847 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:05.380525 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:05.386745 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.387270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.387707 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.388437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.388895 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.389321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.519225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:05.527564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4b1a9545-4aea-4105-961b-b5ab9be64759 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.151733 Sep 11 00:56:05.528131 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 200/390] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:05 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:05.611465 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:05.612255 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:05.619346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.619845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.620286 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.620991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.621446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.622493 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.624221 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-deeac9e9-e3a8-4525-9681-14d925685b85 req-d0711040-daf2-41e8-94af-150062db5d6d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "984863bd-b3a1-4546-8b36-720ba324d7ac", "tag": "883cbb85-cb19-4ae7-86be-1e7abfa4f1f3"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:05.634132 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-deeac9e9-e3a8-4525-9681-14d925685b85 req-d0711040-daf2-41e8-94af-150062db5d6d service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:05.634619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-deeac9e9-e3a8-4525-9681-14d925685b85 req-d0711040-daf2-41e8-94af-150062db5d6d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:05.635086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-deeac9e9-e3a8-4525-9681-14d925685b85 req-d0711040-daf2-41e8-94af-150062db5d6d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:05.646198 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-deeac9e9-e3a8-4525-9681-14d925685b85 req-d0711040-daf2-41e8-94af-150062db5d6d service nova] Creating event network-changed:883cbb85-cb19-4ae7-86be-1e7abfa4f1f3 for instance 984863bd-b3a1-4546-8b36-720ba324d7ac on hv-cinder-maste Sep 11 00:56:05.651973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-deeac9e9-e3a8-4525-9681-14d925685b85 req-d0711040-daf2-41e8-94af-150062db5d6d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031422 Sep 11 00:56:05.652761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 201/391] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:56:05.761132 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:05.769264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3b9930a1-1f27-48b8-821a-abc08f88cebd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.161868 Sep 11 00:56:05.769834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 191/392] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:05 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:06.098961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:06.099664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:06.106285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.106769 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.107228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.107922 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.108373 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.108821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.236671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:06.246175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:06.247633 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:06.248782 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1933168b-beb6-4909-9342-ce5d413f5e30 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.154764 Sep 11 00:56:06.249338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 202/393] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:06 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:06.362505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:06.363090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:06.370284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.370772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.371239 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.371928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.372382 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.372808 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.506043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:06.514223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-68e5b4f4-6647-40fa-840b-c64457ce8198 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.155633 Sep 11 00:56:06.514947 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 192/394] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:06 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:06.551328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:06.551944 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:06.558489 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.559013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.559453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.560147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.560612 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.561036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.689208 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:06.696886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7ba52a5f-1bbc-404c-99bc-bfaea3f1ecdb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.149364 Sep 11 00:56:06.697450 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 203/395] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:06 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:06.794001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:06.794612 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:06.800569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.801057 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.801497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.802189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:06.802645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:06.803114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:06.929889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:06.937492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-21c4f402-9253-4086-97de-28febf940a59 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.147382 Sep 11 00:56:06.938048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 193/396] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:06 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:07.275004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:07.275615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:07.282003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.282479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.282944 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.283624 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.284327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.284745 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.538129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:07.538903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:07.545178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.545667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.546104 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.546819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.547279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.547704 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.550378 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=100 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:07.655570 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:07.665116 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:07.666553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:07.667738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-057b026f-e0af-4df8-9e77-8e6df07fdc03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.397604 Sep 11 00:56:07.668304 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 204/397] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:07 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:07.700242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:07.709095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-89a94d9e-ad4d-4a08-a3ac-66ae29228f66 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.175281 Sep 11 00:56:07.709661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 194/398] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:07 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:07.720198 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:07.720815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:07.726839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.727325 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.727765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.728455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.728910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.729338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.857419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:07.865361 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6f141b84-8b57-4282-bbeb-3e605f881a48 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1357 microversion: 2.1 time: 0.148972 Sep 11 00:56:07.866079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 205/399] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:07 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1357 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:07.960190 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:07.960807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:07.966677 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.967193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.967683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:07.968356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:07.968805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:07.969228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.095554 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:08.104196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-58befb5d-f3a0-49ce-bcd4-56df7840d4b4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.148014 Sep 11 00:56:08.104765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 195/400] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:07 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:08.693665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:08.694875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:08.700898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:08.701379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:08.701818 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.702508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:08.702997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:08.703499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.731637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:08.732265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:08.738646 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:08.739184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:08.739617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.740300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:08.740750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:08.741413 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.879406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:08.888939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:08.890378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:08.891567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e9c35b49-b55c-42be-ab27-e1abfaa4a634 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.202030 Sep 11 00:56:08.892129 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 206/401] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:08 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:08.896455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:08.897100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:08.903499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:08.903987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:08.904421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.905113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:08.905759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:08.905999 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:08.913635 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:08.921958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-05bb120c-0e51-4ecd-8360-be2e646a6553 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.194084 Sep 11 00:56:08.922527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 196/402] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:08 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:09.035815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:09.043903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-971534d5-52f2-432f-a08d-1b6b38f68718 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1547 microversion: 2.1 time: 0.150834 Sep 11 00:56:09.044467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 207/403] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:08 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1547 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:09.067213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:09.067829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:09.074233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.074720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.075293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.075985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.076440 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.076871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.129460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:09.131165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:09.138003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.138498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.139464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.140178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.140640 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.141062 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.257739 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:09.266803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-64d3c008-553e-46f3-8b07-4f8325bd6833 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1547 microversion: 2.1 time: 0.203367 Sep 11 00:56:09.278816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 197/404] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:09 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1547 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:09.290083 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:09.295255 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:09.296336 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Searching by: {'deleted': False, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24'} {{(pid=126979) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 11 00:56:09.303195 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.304588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.305184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.306041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.306697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.307227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.310247 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:09.318662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-da525ab2-bf66-42a0-b8a1-5929bd9df52c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.193149 Sep 11 00:56:09.319452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 208/405] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:09 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:09.400095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.multi_cell_list [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=126979) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 11 00:56:09.494293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1245a63b-01b3-463a-9879-56a5643f19e9 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.209096 Sep 11 00:56:09.494882 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 198/406] 10.222.0.27 () {58 vars in 1125 bytes} [Sun Sep 11 00:56:09 2022] GET /compute/v2.1/servers => generated 342 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:56:09.506404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:09.506991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:09.512993 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.513477 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.513940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.514625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.515107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.515531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.657285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:09.665935 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f920b1a2-ee2c-480e-985e-04a69b72f102 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1547 microversion: 2.1 time: 0.163345 Sep 11 00:56:09.666718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 209/407] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:09 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1547 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:09.917295 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:09.917918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:09.924546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.925575 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.925575 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.926231 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.926694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.927144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.946655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:09.947474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:09.958492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.959372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.965682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:09.966479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:09.967006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:09.967451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:10.117727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:10.126630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:10.128202 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:10.129457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fd42e658-ccb8-4d22-a51b-0933697f5c94 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.216602 Sep 11 00:56:10.130020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 199/408] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:09 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:10.144207 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:10.153219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3956f626-d0d9-4ed0-ae5a-038a495f6f83 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1734 microversion: 2.1 time: 0.211589 Sep 11 00:56:10.153786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 210/409] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:09 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1734 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:10.342011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:10.342542 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:10.348473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:10.348956 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:10.349420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:10.350110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:10.350561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:10.351031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:10.481536 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:10.489182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fd2f9da8-2684-4776-b5d3-e3a9ff0bd2ee tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.151238 Sep 11 00:56:10.489761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 200/410] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:10 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:11.156104 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:11.156818 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.164321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.164815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.165307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.166005 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.166461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.166924 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.176983 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:11.177594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.184870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.185355 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.185784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.186469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.187262 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.187398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.187486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:11.360895 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:11.369702 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:11.371210 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:11.372393 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-dd5e9673-aca1-49f6-8538-5f1df8a3d287 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.220121 Sep 11 00:56:11.372952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 211/411] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:11 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:11.374408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:11.377547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5b00b1ee-8b83-4674-bf79-ae77aef74879"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:11.380000 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.383398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-015844bd-7485-4bce-8aa1-52d3d337e7e8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1714 microversion: 2.1 time: 0.210144 Sep 11 00:56:11.383958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 201/412] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:11 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1714 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:11.386225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.386719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.387185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.387889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.388348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.388776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.501423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:11.501922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.508010 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.508496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.508933 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.509619 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.510072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.510493 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.635102 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] HTTP exception thrown: Cannot 'detach_volume' instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 while it is in vm_state rescued Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Request method failure captured: Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments/2f9be68f-b939-4124-aeea-e4b84fd8b4f7 HTTP/1.1 Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFZLkHjEmrxO7dxRA2K7NCkTkd6TbVWwXHlucfSpPoRvOHHB3oAJBaZcC7qeptnSWmQlNYC7VVp9rstzO1y1WIthC38SS1aTqw9V5bzg9Nair1V4TI7JOjqgjpvcNn4WH05oWOYYYTimbkYeWMgcOfVBndKoCvBDrkrzxnnHc0edIA Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:56:11.636723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_6a58aefc25e141ca93821a80859875e8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 5df6979effd84ece8d75b631ba7da196 Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:56:11.638319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Cannot 'detach_volume' instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 while it is in vm_state rescued Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'server_id': 'eb3cc25c-09a9-47c9-bc2e-d5227478ab70', 'id': '2f9be68f-b939-4124-aeea-e4b84fd8b4f7'} Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:56:11.642486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 while it is in vm_state rescued Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 while it is in vm_state rescued Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:11.644038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Returning 409 to user: Cannot 'detach_volume' instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 while it is in vm_state rescued {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:56:11.645228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-caaafc7d-083a-45cc-b42d-343bc9313ded tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments/2f9be68f-b939-4124-aeea-e4b84fd8b4f7" status: 409 len: 152 microversion: 2.1 time: 0.140691 Sep 11 00:56:11.645228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 202/413] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:56:11 2022] DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments/2f9be68f-b939-4124-aeea-e4b84fd8b4f7 => generated 152 bytes in 142 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 11 00:56:11.647776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:11.648910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.656206 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.656686 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.657117 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.657801 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.658251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.658940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.789527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:11.797706 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d23d8838-f86d-4d78-8edc-3e904fb22f54 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.157393 Sep 11 00:56:11.798366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 203/414] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:11 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:11.807803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:11.807803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.809326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.809814 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.810249 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.810966 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.811423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.811848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.875469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-df728bfb-9940-451e-b8a0-2e859e7a9946 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "POST /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/action" status: 202 len: 0 microversion: 2.1 time: 0.075759 Sep 11 00:56:11.876030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 204/415] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:56:11 2022] POST /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/action => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:56:11.884669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:11.885369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:11.891256 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.891737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.892176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:11.892872 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:11.893328 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:11.893756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.021621 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:12.029801 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-88326681-aab3-4f68-af7a-6da946279849 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1722 microversion: 2.1 time: 0.149471 Sep 11 00:56:12.030355 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 205/416] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:11 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:12.398647 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:12.399295 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:12.405768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.406253 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.406685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.407400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.407854 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.408280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.541455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:12.550002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:12.551525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:12.552692 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-dc698a5b-5398-478b-8db4-647998f437a9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.158801 Sep 11 00:56:12.553905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 206/417] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:12 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:56:12.684006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:12.684458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:12.690397 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.690913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.691351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.692037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.692492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.692916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.715769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a57c9148-fcf5-4bbf-89b3-45ac4c5bc18d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/ips" status: 200 len: 149 microversion: 2.1 time: 0.035599 Sep 11 00:56:12.716316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 207/418] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:56:12 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/ips => generated 149 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:56:12.728447 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "193ee13b-53d8-4301-b1fd-7ec6fce1bdc7"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:12.730853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:12.736505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.736981 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.737410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.738078 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.738525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.738967 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.823298 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Invalid volume: volume 193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 is already attached to instances: fb960ab4-4a34-4074-813f-cd971d916bcf Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: POST /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/os-volume_attachments HTTP/1.1 Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 74 Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:56:12.825287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {"volumeAttachment": {"volumeId": "193ee13b-53d8-4301-b1fd-7ec6fce1bdc7"}} Sep 11 00:56:12.826815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Invalid volume: volume 193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 is already attached to instances: fb960ab4-4a34-4074-813f-cd971d916bcf Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'server_id': '9a843e04-0782-4e1e-b379-9be9a7106442', 'body': {'volumeAttachment': {'volumeId': '193ee13b-53d8-4301-b1fd-7ec6fce1bdc7'}}} Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 is already attached to instances: fb960ab4-4a34-4074-813f-cd971d916bcf Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:12.831326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 is already attached to instances: fb960ab4-4a34-4074-813f-cd971d916bcf Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:12.832742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 is already attached to instances: fb960ab4-4a34-4074-813f-cd971d916bcf Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 400 to user: Invalid volume: volume 193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 is already attached to instances: fb960ab4-4a34-4074-813f-cd971d916bcf {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e9292e19-05bb-45a2-8a61-88f7ebeb9a5d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.103086 Sep 11 00:56:12.834243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 208/419] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:56:12 2022] POST /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442/os-volume_attachments => generated 174 bytes in 104 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 11 00:56:12.835556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:12.835773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:12.842300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.842781 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.843247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.843938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.844393 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.844816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.925571 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-05f1eb49-0ee6-4c91-9cc5-5e8e32c74d3d tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.551406 Sep 11 00:56:12.926228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 212/420] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:56:11 2022] POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments => generated 194 bytes in 1553 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:12.938122 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:12.938920 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:12.947521 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.948072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.948724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.949586 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:12.950203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:12.950866 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:12.991928 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:56:13.027376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:13.036444 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:13.040560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c28ff8c2-ae52-4cf0-9414-a442775f71a0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.211906 Sep 11 00:56:13.041570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 209/421] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:12 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:13.053723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:13.054671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:13.063212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:13.063867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:13.064773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:13.065515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:13.065980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:13.066408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:13.094254 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-51d2cdc8-fcc0-4056-9783-4cfdc379944c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 204 len: 0 microversion: 2.1 time: 0.160488 Sep 11 00:56:13.094853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 213/422] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:56:12 2022] DELETE /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:56:13.103728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:13.104501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:13.110732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:13.111339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:13.111837 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:13.112588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:13.113122 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:13.113604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:13.251660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:13.260526 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-67dd30ed-44fc-4d35-ba62-ecaa8e54273a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1722 microversion: 2.1 time: 0.210841 Sep 11 00:56:13.261097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 210/423] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:13 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1722 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:13.295171 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:13.302910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-27467fd5-4d60-41a9-bbeb-70269193a134 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1682 microversion: 2.1 time: 0.203662 Sep 11 00:56:13.303686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 214/424] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:13 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1682 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:13.578982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:13.579598 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:13.585994 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:13.586474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:13.586937 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:13.587626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:13.588078 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:13.588504 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:13.720768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:13.731405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:13.732841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:13.734034 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b3add8c1-65b8-4ca7-b73b-8fe891754de4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.158952 Sep 11 00:56:13.734578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 211/425] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:13 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:14.067585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:14.068197 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:14.074721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.075232 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.075659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.076338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.076787 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.077205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.206280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:14.214413 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-49cc42f6-4dbc-4d71-8669-4d1374c96fe9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.150644 Sep 11 00:56:14.214999 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 215/426] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:14 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:14.285091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:14.285631 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:14.292418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.292978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.293414 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.294098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.294548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.295000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.325772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:14.326389 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:14.332423 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.332906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.333335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.334069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.335189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.335761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.471686 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:14.480697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7ff45bcb-7aef-45be-9439-405db7f00878 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1722 microversion: 2.1 time: 0.199803 Sep 11 00:56:14.481262 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 212/427] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:14 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1722 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:14.500739 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:14.508423 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bed858f9-303b-4bd6-b20f-cf5611b83c1f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1682 microversion: 2.1 time: 0.186464 Sep 11 00:56:14.508976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 216/428] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:14 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1682 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:14.761833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:14.762442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:14.769388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.769865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.770290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.771079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:14.771533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:14.772619 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:14.901948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:14.910726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:14.912194 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:14.913642 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3e039308-5aef-48bf-9736-7eaf61837a10 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.157121 Sep 11 00:56:14.914197 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 213/429] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:14 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:15.237792 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:15.238482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:15.245511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.246014 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.246451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.247278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.247732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.248346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.423213 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:15.432012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f149ddfa-a71f-4503-aeb9-05a2f6d529c0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.198110 Sep 11 00:56:15.432590 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 217/430] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:15 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:15.503934 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:15.504688 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:15.512756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.512756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.513212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.514029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.514559 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.515098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.531327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:15.532041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:15.538949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.539475 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.539911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.540615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.541068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.541492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.719665 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:15.727467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:15.729004 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-aad78357-ceb0-43e9-8ceb-e5f22c144dde tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1722 microversion: 2.1 time: 0.229474 Sep 11 00:56:15.729570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 214/431] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:15 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1722 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:15.736354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-907af85c-5fea-4c8c-8b07-df5498eabdb1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1682 microversion: 2.1 time: 0.208760 Sep 11 00:56:15.737224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 218/432] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:15 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1682 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:15.940660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:15.941362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:15.949107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.949751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.950323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:15.951330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:15.951899 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:15.952395 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.093120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:16.105019 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:16.106500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:16.107707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-559944d7-b3c9-4109-8c50-9ac0be2b51f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.171743 Sep 11 00:56:16.108263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 215/433] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:15 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:16.429621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-e9d5b75c-d507-44d6-9969-a20227b60d16 req-4035761f-778c-4fa4-9e39-b737d14f9997 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a843e04-0782-4e1e-b379-9be9a7106442", "name": "network-vif-deleted", "tag": "d120b4b9-10bc-4478-8187-9b7014b08cef"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:16.440298 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e9d5b75c-d507-44d6-9969-a20227b60d16 req-4035761f-778c-4fa4-9e39-b737d14f9997 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.440898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e9d5b75c-d507-44d6-9969-a20227b60d16 req-4035761f-778c-4fa4-9e39-b737d14f9997 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.441330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e9d5b75c-d507-44d6-9969-a20227b60d16 req-4035761f-778c-4fa4-9e39-b737d14f9997 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.453923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-e9d5b75c-d507-44d6-9969-a20227b60d16 req-4035761f-778c-4fa4-9e39-b737d14f9997 service nova] Creating event network-vif-deleted:d120b4b9-10bc-4478-8187-9b7014b08cef for instance 9a843e04-0782-4e1e-b379-9be9a7106442 on hv-cinder-maste Sep 11 00:56:16.456672 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:16.457406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:16.459892 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-e9d5b75c-d507-44d6-9969-a20227b60d16 req-4035761f-778c-4fa4-9e39-b737d14f9997 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034145 Sep 11 00:56:16.461049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 219/434] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:16.465168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.465674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.466116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.466847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.467305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.467731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.604351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:16.612236 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-39fec95b-1479-42c4-b0c3-cc4450adc9a3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.159848 Sep 11 00:56:16.613011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 216/435] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:16 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:16.753780 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:16.754622 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:16.761656 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.762177 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.762615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.762991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:16.763340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.763781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.763930 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:16.764486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.770206 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.781879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.781879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.781879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:16.781879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:16.782363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:16.878825 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:16.890550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-26abb80a-9620-4ab5-acc2-204b1e57c098 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1614 microversion: 2.1 time: 0.132328 Sep 11 00:56:16.891252 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 217/436] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:16 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1614 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:16.942828 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:16.952150 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-879235d1-1ee9-415d-ad7a-6f5d0430b6a7 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1722 microversion: 2.1 time: 0.202578 Sep 11 00:56:16.952921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 220/437] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:16 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1722 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:17.134475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:17.135137 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:17.141656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.142143 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.142581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.143310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.143763 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.144189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.291293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:17.301472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:17.304055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:17.305268 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0edad721-10b1-48ec-83cd-fa4ce312d9ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.175500 Sep 11 00:56:17.319276 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 218/438] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:17 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:17.452491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-6c055db6-cade-4b12-b90c-616c7747dea3 req-87979d5f-b3bd-4ed2-9142-2034188166e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "name": "network-vif-plugged", "status": "completed", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:17.458024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:17.468735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6c055db6-cade-4b12-b90c-616c7747dea3 req-87979d5f-b3bd-4ed2-9142-2034188166e9 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.469230 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6c055db6-cade-4b12-b90c-616c7747dea3 req-87979d5f-b3bd-4ed2-9142-2034188166e9 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.469655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6c055db6-cade-4b12-b90c-616c7747dea3 req-87979d5f-b3bd-4ed2-9142-2034188166e9 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.482410 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-6c055db6-cade-4b12-b90c-616c7747dea3 req-87979d5f-b3bd-4ed2-9142-2034188166e9 service nova] Creating event network-vif-plugged:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:56:17.487851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-6c055db6-cade-4b12-b90c-616c7747dea3 req-87979d5f-b3bd-4ed2-9142-2034188166e9 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039011 Sep 11 00:56:17.488602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 221/439] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:56:17.635381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:17.636177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:17.642395 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.642917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.643358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.644054 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.644512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.644940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.797492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:17.806418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7755b0db-2140-447b-9811-bbc7d6418fb5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.175458 Sep 11 00:56:17.807022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 219/440] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:17 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:17.915575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:17.916200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:17.922355 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.922871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.923308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.924000 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.924460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.924887 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.956814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.957287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.957867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.958381 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'fault' on Instance uuid 9a843e04-0782-4e1e-b379-9be9a7106442 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:56:17.976537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:17.977382 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:17.983621 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.984348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.984785 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:17.985552 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:17.986012 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:17.986594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:18.007678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:18.015958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b82c9ebe-5c93-4b3e-97f6-74dcdee08c51 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1332 microversion: 2.1 time: 0.105458 Sep 11 00:56:18.016626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 222/441] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:17 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1332 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:18.127530 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:18.136239 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-14e3a7f0-a18f-45f4-8f62-9715f07202df tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1722 microversion: 2.1 time: 0.164042 Sep 11 00:56:18.136901 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 220/442] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:17 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:18.333250 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:18.333860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:18.340903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:18.341401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:18.341838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:18.342781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:18.343257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:18.343879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:18.486144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:18.495581 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:18.497095 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:18.498215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a32a243c-772e-4f54-b8b8-6f17c5ae06f9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.169003 Sep 11 00:56:18.498778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 223/443] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:18 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:18.829192 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:18.829830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:18.836200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:18.836683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:18.837115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:18.837827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:18.838279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:18.838701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:18.966564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:18.974288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0aaedab5-2505-482e-a1d1-eca0e1cc9758 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.148995 Sep 11 00:56:18.974815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 221/444] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:18 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:19.041451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:19.042060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:19.048209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.048687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.049114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.049791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.050235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.050653 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.083928 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.084406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.085134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.085649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'fault' on Instance uuid 9a843e04-0782-4e1e-b379-9be9a7106442 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:56:19.131286 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:19.139463 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bd89c7ed-81ec-4f73-8d25-13154701eedd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1332 microversion: 2.1 time: 0.102616 Sep 11 00:56:19.140241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 224/445] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:19 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1332 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:19.161678 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:19.162060 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:19.168961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.169455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.169894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.170636 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.171117 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.171595 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.307029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:19.315228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c728cbae-e0c7-4737-9dcd-9982e8dcf784 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1728 microversion: 2.1 time: 0.158276 Sep 11 00:56:19.315790 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 222/446] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:19 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1728 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:19.338306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:19.338832 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:19.344916 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.345488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.345816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.346501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.346983 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.347410 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.423242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-70f8eefe-4180-40e7-bd18-70eba4d20311 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.088705 Sep 11 00:56:19.423825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 225/447] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 00:56:19 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-interface => generated 284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:19.525120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:19.525832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:19.527732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:19.532998 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.533423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.534129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.534850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:19.535310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:19.535736 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:19.693837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:19.703687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:19.705301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:19.706935 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b77ee40d-f0cb-452a-9821-96aae7a3b8df tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.186136 Sep 11 00:56:19.716519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 223/448] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:19 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:19.997884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:19.998544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:20.004772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.005264 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.005701 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.007013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.007477 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.007897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.012170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:20.147291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:20.155598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a75a2dd8-5c49-4ca4-9815-e0a359e0d492 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.162105 Sep 11 00:56:20.156354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 226/449] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:19 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:20.175911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:20.176525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:20.183751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.184237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.184668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.185352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.185799 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.186219 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.221671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.222155 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.222704 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.223571 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'fault' on Instance uuid 9a843e04-0782-4e1e-b379-9be9a7106442 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:56:20.270036 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:20.278980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5749cc83-f5d7-486d-ac3e-a11e0be182db tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 200 len: 1332 microversion: 2.1 time: 0.106779 Sep 11 00:56:20.280130 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 224/450] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:20 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 1332 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:20.404322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-6b9022f0-ae35-4956-ba0d-4a9470b0fd50 req-932ea1a0-382b-44e4-85d2-8f7482428e2f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb3cc25c-09a9-47c9-bc2e-d5227478ab70", "tag": "f4ee5962-5a14-4994-83b5-3235d55d032f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:20.416202 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6b9022f0-ae35-4956-ba0d-4a9470b0fd50 req-932ea1a0-382b-44e4-85d2-8f7482428e2f service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.416762 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6b9022f0-ae35-4956-ba0d-4a9470b0fd50 req-932ea1a0-382b-44e4-85d2-8f7482428e2f service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.417270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6b9022f0-ae35-4956-ba0d-4a9470b0fd50 req-932ea1a0-382b-44e4-85d2-8f7482428e2f service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.428475 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-6b9022f0-ae35-4956-ba0d-4a9470b0fd50 req-932ea1a0-382b-44e4-85d2-8f7482428e2f service nova] Creating event network-changed:f4ee5962-5a14-4994-83b5-3235d55d032f for instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 on hv-cinder-maste Sep 11 00:56:20.434705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-6b9022f0-ae35-4956-ba0d-4a9470b0fd50 req-932ea1a0-382b-44e4-85d2-8f7482428e2f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034298 Sep 11 00:56:20.435343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 227/451] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:20.733223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:20.733832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:20.739820 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.740752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.742352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.743099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:20.743559 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:20.743984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:20.882848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:20.891745 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:20.893887 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:20.895396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a25b5e1d-423d-4f40-99bb-75737d610e0a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.166088 Sep 11 00:56:20.896030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 225/452] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:20 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:21.180918 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:21.181623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:21.188029 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:21.188520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:21.188951 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:21.189649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:21.190101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:21.190523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:21.303961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:21.304721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:21.311101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:21.311715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:21.312258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:21.313069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:21.313622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:21.314166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:21.318807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=100 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:21.328175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:21.329570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 HTTP/1.1 Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:56:21.331437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:56:21.332970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '9a843e04-0782-4e1e-b379-9be9a7106442'} Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:56:21.333759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:56:21.335366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:56:21.337246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:56:21.338223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0554c162-f460-417c-a906-bfd6cf14a5cd tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 404 len: 111 microversion: 2.1 time: 0.039041 Sep 11 00:56:21.338903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 226/453] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:21 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 00:56:21.342354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e2b74184-6006-4c08-90ba-fa137b68bda5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.163664 Sep 11 00:56:21.342354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 228/454] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:21 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:21.353376 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-341ddded-6e16-4e2c-88ac-9a8daf91a03d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '.version_select at 0x7fc01af59310>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:21.380619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-341ddded-6e16-4e2c-88ac-9a8daf91a03d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1845430137" status: 202 len: 0 microversion: 2.1 time: 0.031810 Sep 11 00:56:21.385365 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 229/455] 10.222.0.27 () {58 vars in 1221 bytes} [Sun Sep 11 00:56:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1845430137 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:56:21.926324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:21.927040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:21.933946 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:21.934497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:21.934931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:21.935678 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:21.936139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:21.936572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.101769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:22.117048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:22.118774 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:22.122121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-db10d7c6-bc42-409f-861a-21606d0928f8 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1807 microversion: 2.20 time: 0.200254 Sep 11 00:56:22.123235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 227/456] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:21 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1807 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:22.165373 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:22.165880 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:22.171885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.172371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.172795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.173668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.174279 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.174881 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.314853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'services' on Instance uuid fb960ab4-4a34-4074-813f-cd971d916bcf {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:56:22.365887 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:22.366328 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:22.373105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.374040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.374752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.375696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.376295 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.377083 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.452228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-15db3ca5-6ec4-44ee-a28c-354c0b6f7a14 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments/193ee13b-53d8-4301-b1fd-7ec6fce1bdc7" status: 202 len: 0 microversion: 2.1 time: 0.290892 Sep 11 00:56:22.453464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 230/457] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:56:22 2022] DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments/193ee13b-53d8-4301-b1fd-7ec6fce1bdc7 => generated 0 bytes in 293 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:56:22.459820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-6ba5cfdb-0dc0-4715-a23a-011883a064f6 req-a452257f-0046-4cfa-86dd-42456ebb9dc4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb3cc25c-09a9-47c9-bc2e-d5227478ab70", "tag": "f4ee5962-5a14-4994-83b5-3235d55d032f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:22.470057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6ba5cfdb-0dc0-4715-a23a-011883a064f6 req-a452257f-0046-4cfa-86dd-42456ebb9dc4 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.470913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6ba5cfdb-0dc0-4715-a23a-011883a064f6 req-a452257f-0046-4cfa-86dd-42456ebb9dc4 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.471507 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6ba5cfdb-0dc0-4715-a23a-011883a064f6 req-a452257f-0046-4cfa-86dd-42456ebb9dc4 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.484738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-6ba5cfdb-0dc0-4715-a23a-011883a064f6 req-a452257f-0046-4cfa-86dd-42456ebb9dc4 service nova] Creating event network-changed:f4ee5962-5a14-4994-83b5-3235d55d032f for instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 on hv-cinder-maste Sep 11 00:56:22.490231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-6ba5cfdb-0dc0-4715-a23a-011883a064f6 req-a452257f-0046-4cfa-86dd-42456ebb9dc4 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035166 Sep 11 00:56:22.490897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 231/458] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:22.496334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:22.499079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:22.505676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.506364 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.506872 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.507579 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:22.508028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:22.508630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:22.552472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:22.562218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e317404f-c8b4-4418-ad40-940e296ab4af tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070127 Sep 11 00:56:22.563163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 232/459] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:56:22 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:22.563849 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-011f5b32-89ee-4b75-af0f-f9ccfebdddc9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.201709 Sep 11 00:56:22.570821 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 228/460] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:22 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:23.149942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:23.150543 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:23.157155 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.157632 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.158059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.159024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.159611 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.160113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.292811 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:23.301358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:23.303330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:23.305321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1e8bec11-dc19-4d63-a678-2ad1def4946a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1794 microversion: 2.20 time: 0.160637 Sep 11 00:56:23.306186 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 233/461] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:56:23 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1794 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:56:23.576294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:23.578750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:23.586821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:23.587494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:23.594498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.595016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.595485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.600201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.600693 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.601127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.601827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.602281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.602711 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.606835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.607584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.608178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.641395 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-47ede034-b18a-442c-aad8-a1873bb4feab tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069369 Sep 11 00:56:23.646683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 229/462] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:56:23 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:23.769320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-fcab6db7-93a3-48ee-88a1-caf74f0d56db req-84d9aa08-f4e1-42d3-9545-8d299c2b0433 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "984863bd-b3a1-4546-8b36-720ba324d7ac", "name": "network-vif-plugged", "status": "completed", "tag": "883cbb85-cb19-4ae7-86be-1e7abfa4f1f3"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:23.770201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:23.780249 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-fcab6db7-93a3-48ee-88a1-caf74f0d56db req-84d9aa08-f4e1-42d3-9545-8d299c2b0433 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.780734 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-fcab6db7-93a3-48ee-88a1-caf74f0d56db req-84d9aa08-f4e1-42d3-9545-8d299c2b0433 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.781162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-fcab6db7-93a3-48ee-88a1-caf74f0d56db req-84d9aa08-f4e1-42d3-9545-8d299c2b0433 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.784447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3cdad74d-446f-448f-b933-bbfcd3d2387a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.195600 Sep 11 00:56:23.788006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 234/463] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:23 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:23.793800 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-fcab6db7-93a3-48ee-88a1-caf74f0d56db req-84d9aa08-f4e1-42d3-9545-8d299c2b0433 service nova] Creating event network-vif-plugged:883cbb85-cb19-4ae7-86be-1e7abfa4f1f3 for instance 984863bd-b3a1-4546-8b36-720ba324d7ac on hv-cinder-maste Sep 11 00:56:23.795159 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:23.795967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:23.803708 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.804069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-fcab6db7-93a3-48ee-88a1-caf74f0d56db req-84d9aa08-f4e1-42d3-9545-8d299c2b0433 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052980 Sep 11 00:56:23.804451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.804993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 230/464] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:56:23.805113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.805614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:23.806064 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:23.806761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:23.940403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:23.949789 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4fa24b09-7c6e-4c54-98c6-2cad8e838758 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1593 microversion: 2.1 time: 0.159991 Sep 11 00:56:23.962942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 235/465] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:23 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1593 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:24.654234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:24.656482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:24.664044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:24.664533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:24.664971 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:24.665668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:24.666124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:24.666549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:24.696735 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7c68ad18-fca6-4141-8a7a-e533477288c4 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046739 Sep 11 00:56:24.697276 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 231/466] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:56:24 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:24.808081 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:24.808633 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:24.815075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:24.815560 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:24.815998 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:24.816683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:24.817130 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:24.817547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.037119 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:25.045679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a4de8518-dc41-43ca-83e1-7ab52df1c06b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.241520 Sep 11 00:56:25.046247 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 236/467] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:24 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:25.710570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:25.712942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:25.719834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:25.720545 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:25.721145 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.721985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:25.722585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:25.723183 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.753862 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ef643af0-845a-4cda-b2fe-77b17f786ef5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048386 Sep 11 00:56:25.754414 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 232/468] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:56:25 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 00:56:25.837100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:25.837871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:25.844479 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:25.845086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:25.845518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.846233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:25.846673 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:25.847269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.874952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:56:25.962427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-45dd0fac-9e6f-4501-803f-43aa348f44e5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 204 len: 0 microversion: 2.1 time: 0.129445 Sep 11 00:56:25.963686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 237/469] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:56:25 2022] DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:56:25.972427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:25.973327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:25.980342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:25.980835 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:25.981843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.982555 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:25.983178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:25.983740 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:25.999319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-c75d5cf3-58ea-46de-85bb-1e0bb6918469 req-32cd865a-76a9-4e2b-b340-cf8b946bd898 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:26.011971 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-c75d5cf3-58ea-46de-85bb-1e0bb6918469 req-32cd865a-76a9-4e2b-b340-cf8b946bd898 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:26.012455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-c75d5cf3-58ea-46de-85bb-1e0bb6918469 req-32cd865a-76a9-4e2b-b340-cf8b946bd898 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:26.013235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-c75d5cf3-58ea-46de-85bb-1e0bb6918469 req-32cd865a-76a9-4e2b-b340-cf8b946bd898 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:26.027650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-c75d5cf3-58ea-46de-85bb-1e0bb6918469 req-32cd865a-76a9-4e2b-b340-cf8b946bd898 service nova] Creating event network-changed:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:56:26.036569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-c75d5cf3-58ea-46de-85bb-1e0bb6918469 req-32cd865a-76a9-4e2b-b340-cf8b946bd898 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041574 Sep 11 00:56:26.037845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 238/470] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:56:26.073806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:26.074721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:26.085275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:26.086103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:26.086664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:26.087490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:26.088070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:26.088593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:26.155809 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:26.169416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6ea84233-fa73-457d-9a78-847be4dd0323 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1682 microversion: 2.1 time: 0.201541 Sep 11 00:56:26.170418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 233/471] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:25 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1682 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:26.257639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:26.265296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-86a66fc2-8b18-41ad-88bb-8c2d38a61d48 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1264 microversion: 2.1 time: 0.199856 Sep 11 00:56:26.277040 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 239/472] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:26 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1264 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:27.031953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-674780763"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:27.032487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:27.040134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.040677 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.041154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.041847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.042278 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.042727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.175964 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.network.security_group_api [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Adding security group c7087b87-7128-4e1c-9bb8-3d984694865c to port 497f3d01-c618-4bc4-8866-0f2093427106 Sep 11 00:56:27.194278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:27.194936 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:27.201556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.202040 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.202845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.203734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.204240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.204667 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.446415 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:27.454105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-14ad31b1-8e14-4c67-b975-ec366a680608 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1682 microversion: 2.1 time: 0.263684 Sep 11 00:56:27.454684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 240/473] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:27 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1682 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:27.475772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:27.476395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:27.482219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.482918 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.483355 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.484042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.484491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.484910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.669240 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fa4fba73-1496-49c4-987f-1a7bc0c8ea83 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/action" status: 202 len: 0 microversion: 2.1 time: 0.641205 Sep 11 00:56:27.670094 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 234/474] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:56:27 2022] POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/action => generated 0 bytes in 643 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:56:27.678577 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:27.679308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:27.685614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.686177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.686684 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.687477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.688027 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.688523 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.698014 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:27.707203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-97b0ee74-2dbe-447d-9eec-36399660ad50 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1451 microversion: 2.1 time: 0.235220 Sep 11 00:56:27.708246 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 241/475] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:27 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1451 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:27.733484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:27.734413 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:27.743168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.743678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.744152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.745014 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:27.745471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:27.746105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:27.916152 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:27.925066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c39511a5-7ee9-40c9-9d6c-e9808c3bb19a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1639 microversion: 2.1 time: 0.250635 Sep 11 00:56:27.939181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 235/476] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:27 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1639 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:27.944494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:27.953854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-66afa2c6-e51e-4dc3-8c2e-daba5ed9ae67 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1451 microversion: 2.1 time: 0.224951 Sep 11 00:56:27.954432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 242/477] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:27 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1451 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:28.055578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-c24429a9-dcb0-43aa-b147-449ece642edd req-f5c9213b-1d91-4088-b20e-7358b1863c31 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:28.066572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c24429a9-dcb0-43aa-b147-449ece642edd req-f5c9213b-1d91-4088-b20e-7358b1863c31 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:28.067435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c24429a9-dcb0-43aa-b147-449ece642edd req-f5c9213b-1d91-4088-b20e-7358b1863c31 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:28.068033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-c24429a9-dcb0-43aa-b147-449ece642edd req-f5c9213b-1d91-4088-b20e-7358b1863c31 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:28.095465 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-c24429a9-dcb0-43aa-b147-449ece642edd req-f5c9213b-1d91-4088-b20e-7358b1863c31 service nova] Creating event network-changed:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:56:28.108576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-c24429a9-dcb0-43aa-b147-449ece642edd req-f5c9213b-1d91-4088-b20e-7358b1863c31 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056882 Sep 11 00:56:28.109182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 236/478] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:28.478679 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:28.479349 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:28.486488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:28.487025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:28.487471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:28.488170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:28.488625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:28.489050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:28.510569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:28.716615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:28.725111 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e598bdbf-88ff-4fc8-bc41-d7f3e76945bf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1682 microversion: 2.1 time: 0.250133 Sep 11 00:56:28.725803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 243/479] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:28 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1682 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:29.748698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:29.749409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:29.756596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=100 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:29.758955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:29.759604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:29.760211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:29.761084 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:29.761690 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:29.762531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:29.998593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:30.014051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-54682164-efc5-493d-ae78-3d3a6928097b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1682 microversion: 2.1 time: 0.269395 Sep 11 00:56:30.015440 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 237/480] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:29 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1682 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:30.125957 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-ba851911-71bc-4546-abd5-612c34819f42 req-1b8d0555-cc6a-4585-8396-287dec5a95f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "984863bd-b3a1-4546-8b36-720ba324d7ac", "tag": "883cbb85-cb19-4ae7-86be-1e7abfa4f1f3"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:30.136744 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ba851911-71bc-4546-abd5-612c34819f42 req-1b8d0555-cc6a-4585-8396-287dec5a95f0 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:30.137228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ba851911-71bc-4546-abd5-612c34819f42 req-1b8d0555-cc6a-4585-8396-287dec5a95f0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:30.137658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ba851911-71bc-4546-abd5-612c34819f42 req-1b8d0555-cc6a-4585-8396-287dec5a95f0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:30.149850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-ba851911-71bc-4546-abd5-612c34819f42 req-1b8d0555-cc6a-4585-8396-287dec5a95f0 service nova] Creating event network-changed:883cbb85-cb19-4ae7-86be-1e7abfa4f1f3 for instance 984863bd-b3a1-4546-8b36-720ba324d7ac on hv-cinder-maste Sep 11 00:56:30.155209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-ba851911-71bc-4546-abd5-612c34819f42 req-1b8d0555-cc6a-4585-8396-287dec5a95f0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032869 Sep 11 00:56:30.155884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 244/481] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:31.041182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:31.041969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:31.056585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:31.057071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:31.057689 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:31.058398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:31.059191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:31.059621 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:31.131652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:31.140049 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0ed2b255-5829-4387-bfb4-050bc25c66f7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1614 microversion: 2.1 time: 0.103892 Sep 11 00:56:31.146877 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 238/482] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:31 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1614 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:31.317291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-93848846-f797-40b7-9751-857f7c7f299c req-6571db0a-f889-4721-befd-a1a0c332bced service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fb960ab4-4a34-4074-813f-cd971d916bcf", "name": "network-vif-deleted", "tag": "41d6739a-7225-40b2-aedd-df0beff65d5f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:31.328071 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-93848846-f797-40b7-9751-857f7c7f299c req-6571db0a-f889-4721-befd-a1a0c332bced service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:31.328560 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-93848846-f797-40b7-9751-857f7c7f299c req-6571db0a-f889-4721-befd-a1a0c332bced service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:31.328993 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-93848846-f797-40b7-9751-857f7c7f299c req-6571db0a-f889-4721-befd-a1a0c332bced service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:31.341790 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-93848846-f797-40b7-9751-857f7c7f299c req-6571db0a-f889-4721-befd-a1a0c332bced service nova] Creating event network-vif-deleted:41d6739a-7225-40b2-aedd-df0beff65d5f for instance fb960ab4-4a34-4074-813f-cd971d916bcf on hv-cinder-maste Sep 11 00:56:31.346573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-93848846-f797-40b7-9751-857f7c7f299c req-6571db0a-f889-4721-befd-a1a0c332bced service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033939 Sep 11 00:56:31.347216 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 245/483] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:32.164488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:32.165457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:32.171978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:32.172458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:32.173308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:32.174340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:32.175600 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:32.176266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:32.213954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:32.214442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:32.214914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:32.215381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'fault' on Instance uuid fb960ab4-4a34-4074-813f-cd971d916bcf {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:56:32.258173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:32.267389 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-df30ff6b-e3f4-4b56-bbcf-85e7132c8ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 200 len: 1331 microversion: 2.1 time: 0.106945 Sep 11 00:56:32.279719 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 239/484] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:32 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 1331 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:33.293663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:33.294272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:33.300320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:33.300793 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:33.301218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:33.301895 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:33.303072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:33.303768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:33.316545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf HTTP/1.1 Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:56:33.318283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:56:33.319835 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'fb960ab4-4a34-4074-813f-cd971d916bcf'} Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:56:33.320801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:56:33.322319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:56:33.323842 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:56:33.323842 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-68a7d927-f263-4665-a95f-681654caa394 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 404 len: 111 microversion: 2.1 time: 0.030952 Sep 11 00:56:33.323842 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 246/485] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:33 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:56:33.328520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7981f8e4-567e-49ea-a842-a62058027b82 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '.version_select at 0x7fc01aa30550>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:33.348549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7981f8e4-567e-49ea-a842-a62058027b82 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1190363958" status: 202 len: 0 microversion: 2.1 time: 0.023776 Sep 11 00:56:33.349900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 240/486] 10.222.0.27 () {58 vars in 1221 bytes} [Sun Sep 11 00:56:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1190363958 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:56:33.984244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0d1fb38e-7576-4f39-864f-00efdcc537ea tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: .version_select at 0x7fc01ac58040>, body: {"keypair": {"name": "tempest-keypair-1494663280", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD+gzUh51brShPU/kITZggCCy145Jnp/DzXyNEJdZ1WPUQa+0IWZ1zVuKna//pcZDCDuSyQ4XO9NOAVUCopltJ6Wg7acvNJX3G16y+FR+OxYMTQiFFC4BoVbZSuUvkOCDA=="}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:33.999604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-0d1fb38e-7576-4f39-864f-00efdcc537ea tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:56:34.003458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-0d1fb38e-7576-4f39-864f-00efdcc537ea tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:56:34.025207 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0d1fb38e-7576-4f39-864f-00efdcc537ea tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.044800 Sep 11 00:56:34.025820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 247/487] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:56:33 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:35.287984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-923589759", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e5ef326a-1def-4c9c-ad55-20a6d9fdbde1"}], "security_groups": [{"name": "tempest-securitygroup--1035801087"}], "key_name": "tempest-keypair-1494663280", "user_data": "}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:35.388047 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] validate_networks() for [('e5ef326a-1def-4c9c-ad55-20a6d9fdbde1', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:56:35.750269 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:56:35.750692 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:56:35.751218 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:56:35.751621 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:56:35.752031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:56:35.761450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:35.761923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:35.762349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:35.763134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:35.763580 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:35.763996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:35.781308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:56:35.785370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:56:35.796211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:56:35.798423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:56:35.846724 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8af5b037-3fc4-42e4-91f3-696bee85d95c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.564683 Sep 11 00:56:35.847400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 241/488] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:56:35 2022] POST /compute/v2.1/servers => generated 402 bytes in 566 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:56:35.859303 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1ac47c12-c335-4e28-be39-ea50acc8f444 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:35.859909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1ac47c12-c335-4e28-be39-ea50acc8f444 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:35.915916 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1ac47c12-c335-4e28-be39-ea50acc8f444 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:35.922923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1ac47c12-c335-4e28-be39-ea50acc8f444 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1241 microversion: 2.1 time: 0.068024 Sep 11 00:56:35.923490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 248/489] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:35 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1241 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:36.946123 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:36.946659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:36.952698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:36.953181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:36.953616 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:36.954331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:36.954780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:36.955224 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:37.016830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:37.025324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d8dfbee0-e346-498c-a710-98d3a035b125 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1293 microversion: 2.1 time: 0.083784 Sep 11 00:56:37.025896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 242/490] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:36 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1293 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:38.049336 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:38.050224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:38.057620 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:38.058205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:38.058705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:38.060309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:38.061056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:38.061742 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:38.216744 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:38.236654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1b1e16df-4213-4192-adc6-5cbce4080fdc tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.191127 Sep 11 00:56:38.237214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 249/491] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:38 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:39.021970 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-802f09a8-ae8c-40af-97ce-47388e55f039 req-b7b40766-ec15-4fd1-9aae-5a8d09350eea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650", "tag": "6a7c8c37-25f4-480e-a003-6a8f1f5f186d"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:39.033896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-802f09a8-ae8c-40af-97ce-47388e55f039 req-b7b40766-ec15-4fd1-9aae-5a8d09350eea service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:39.034632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-802f09a8-ae8c-40af-97ce-47388e55f039 req-b7b40766-ec15-4fd1-9aae-5a8d09350eea service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:39.035275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-802f09a8-ae8c-40af-97ce-47388e55f039 req-b7b40766-ec15-4fd1-9aae-5a8d09350eea service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:39.046760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-802f09a8-ae8c-40af-97ce-47388e55f039 req-b7b40766-ec15-4fd1-9aae-5a8d09350eea service nova] Creating event network-changed:6a7c8c37-25f4-480e-a003-6a8f1f5f186d for instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 on hv-cinder-maste Sep 11 00:56:39.051865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-802f09a8-ae8c-40af-97ce-47388e55f039 req-b7b40766-ec15-4fd1-9aae-5a8d09350eea service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034104 Sep 11 00:56:39.053021 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 243/492] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:39.261009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:39.261703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:39.268040 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:39.268520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:39.268945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:39.269634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:39.270085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:39.270505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:39.399124 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:39.406831 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-36bd2a12-2696-472d-a19b-b359ade654c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.149701 Sep 11 00:56:39.407399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 250/493] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:39 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:40.430848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:40.431474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:40.438574 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:40.439090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:40.439521 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:40.440222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:40.440675 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:40.441089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:40.573847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:40.581411 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5724d067-1fa3-451a-b2b6-2a4dd8a7c6d7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.155604 Sep 11 00:56:40.581969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 244/494] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:40 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:41.604130 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:41.604746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:41.611327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:41.611806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:41.612237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:41.612917 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:41.613365 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:41.613785 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:41.743959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:41.752163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-28863116-0d95-4d4c-a77c-8b91a5efee05 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.151921 Sep 11 00:56:41.752752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 251/495] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:41 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:42.776506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:42.777121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:42.783789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:42.784282 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:42.784875 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:42.786099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:42.787568 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:42.788353 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:42.916972 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:42.924561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e9c30d53-0881-4721-9d4e-cb79ecfd952b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.151906 Sep 11 00:56:42.925124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 245/496] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:42 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:43.947397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:43.947941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:43.954354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:43.954977 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:43.955414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:43.956101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:43.956556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:43.956980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:44.097458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:44.106640 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5ded542f-6b0b-41ba-accd-21ef6a0b4003 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.163098 Sep 11 00:56:44.107630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 252/497] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:43 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:45.131077 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:45.131759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:45.138098 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:45.138603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:45.139062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:45.139738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:45.140186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:45.140604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:45.270214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:45.277948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6835ff15-c0a9-4195-8eb1-1e693c8c0381 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.151629 Sep 11 00:56:45.278506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 246/498] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:45 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:45.646940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-55b9cd44-440d-4153-bf0f-63a32d96be17 req-83d5e7b9-db66-4eb9-85d4-4d80983a3195 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650", "name": "network-vif-plugged", "status": "completed", "tag": "6a7c8c37-25f4-480e-a003-6a8f1f5f186d"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:45.657320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-55b9cd44-440d-4153-bf0f-63a32d96be17 req-83d5e7b9-db66-4eb9-85d4-4d80983a3195 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:45.657859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-55b9cd44-440d-4153-bf0f-63a32d96be17 req-83d5e7b9-db66-4eb9-85d4-4d80983a3195 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:45.658289 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-55b9cd44-440d-4153-bf0f-63a32d96be17 req-83d5e7b9-db66-4eb9-85d4-4d80983a3195 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:45.679682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-55b9cd44-440d-4153-bf0f-63a32d96be17 req-83d5e7b9-db66-4eb9-85d4-4d80983a3195 service nova] Creating event network-vif-plugged:6a7c8c37-25f4-480e-a003-6a8f1f5f186d for instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 on hv-cinder-maste Sep 11 00:56:45.684887 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-55b9cd44-440d-4153-bf0f-63a32d96be17 req-83d5e7b9-db66-4eb9-85d4-4d80983a3195 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041858 Sep 11 00:56:45.685561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 253/499] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:46.301538 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:46.303344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:46.305421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:46.306184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:46.312738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:46.313218 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:46.313646 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:46.314329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:46.314773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:46.315211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:46.319831 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=100 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:46.460784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:46.468944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-eb52f667-907f-4d3a-aa93-c4187f52ff5f tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.171416 Sep 11 00:56:46.469509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 247/500] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:46 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:47.490445 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:47.495188 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:47.495896 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:47.497704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=87,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:47.503759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:47.504245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:47.505296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:47.506100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:47.506559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:47.507133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:47.639687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:47.647528 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-22d8d8a4-9a91-4705-ae81-1ef8cd34e02e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1367 microversion: 2.1 time: 0.158585 Sep 11 00:56:47.648084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 254/501] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:47 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:56:48.671081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:48.671693 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:48.678681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:48.679195 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:48.679628 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:48.680316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:48.680771 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:48.681191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:48.810459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:56:48.818639 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0b75cc5d-3256-4bab-85aa-3eea5c1be94a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1557 microversion: 2.1 time: 0.151363 Sep 11 00:56:48.819226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 248/502] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:56:48 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1557 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:56:48.840624 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:56:48.841141 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:56:48.847508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:48.847991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:48.848471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:48.849459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:48.849914 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:48.850343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:48.926332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-441e3cd2-a9c8-4190-8239-ad1be150590d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089673 Sep 11 00:56:48.927472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 255/503] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 00:56:48 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:56:49.850653 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-bd3bdf92-676e-4e96-8839-31af0f67efe9 req-a3533f6e-4735-4b6f-b8d0-94a7fef39089 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650", "tag": "6a7c8c37-25f4-480e-a003-6a8f1f5f186d"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:56:49.860917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-bd3bdf92-676e-4e96-8839-31af0f67efe9 req-a3533f6e-4735-4b6f-b8d0-94a7fef39089 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:56:49.861398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-bd3bdf92-676e-4e96-8839-31af0f67efe9 req-a3533f6e-4735-4b6f-b8d0-94a7fef39089 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:56:49.861834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-bd3bdf92-676e-4e96-8839-31af0f67efe9 req-a3533f6e-4735-4b6f-b8d0-94a7fef39089 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:56:49.873795 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-bd3bdf92-676e-4e96-8839-31af0f67efe9 req-a3533f6e-4735-4b6f-b8d0-94a7fef39089 service nova] Creating event network-changed:6a7c8c37-25f4-480e-a003-6a8f1f5f186d for instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 on hv-cinder-maste Sep 11 00:56:49.879124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-bd3bdf92-676e-4e96-8839-31af0f67efe9 req-a3533f6e-4735-4b6f-b8d0-94a7fef39089 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032861 Sep 11 00:56:49.880448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 249/504] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:56:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:00.160799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:00.161488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=44 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:00.166148 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:00.168362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:00.175061 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:00.175541 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:00.175975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:00.176655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:00.177105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:00.177639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:00.247393 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e852f23c-d4f4-4e09-84d7-532c4be876f8 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/action" status: 202 len: 0 microversion: 2.1 time: 0.087802 Sep 11 00:57:00.248026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 256/505] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:57:00 2022] POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/action => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:57:00.258780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:00.259507 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:00.262866 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=41,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:00.263491 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=13 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:00.268449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:00.268932 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:00.269367 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:00.272328 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:00.272964 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:00.273424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:00.419703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:00.429083 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-54d795a7-d884-414b-946e-3cdee9e83114 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1750 microversion: 2.1 time: 0.176451 Sep 11 00:57:00.429844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 250/506] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:00 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1750 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:01.454464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:01.455191 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:01.461425 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:01.461937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:01.462370 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:01.463094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:01.463544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:01.463965 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:01.556421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:01.556936 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:01.563727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:01.564218 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:01.564655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:01.565344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:01.565793 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:01.566214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:01.609124 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:01.617757 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-96fdece8-4215-4228-b353-e868a1cd7bcd tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.167983 Sep 11 00:57:01.618320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 257/507] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:01 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:01.648352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lazy-loading 'services' on Instance uuid eb3cc25c-09a9-47c9-bc2e-d5227478ab70 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:01.760028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-817753b5-a1ce-4ef3-8afe-deb297052cac tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments/2f9be68f-b939-4124-aeea-e4b84fd8b4f7" status: 202 len: 0 microversion: 2.1 time: 0.208175 Sep 11 00:57:01.760615 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 251/508] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:57:01 2022] DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments/2f9be68f-b939-4124-aeea-e4b84fd8b4f7 => generated 0 bytes in 210 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 11 00:57:01.769029 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:01.771415 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:01.778369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:01.778893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:01.779326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:01.780043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:01.780487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:01.780904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:01.812626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f9263044-6a14-4aeb-bd4d-06627b4f08c5 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047445 Sep 11 00:57:01.813182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 258/509] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:01 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:02.641193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:02.641894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:02.648426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:02.648913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:02.649342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:02.650113 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:02.650567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:02.651009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:02.798454 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:02.807441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-499dd4ed-43ca-4eb1-89a0-f41f335d4382 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.170448 Sep 11 00:57:02.808000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 252/510] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:02 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:02.825240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:02.827416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:02.834144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:02.834627 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:02.835098 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:02.835798 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:02.836253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:02.836676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:02.866140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2f8506af-79b5-4e43-bcb2-dd5b61349d94 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044844 Sep 11 00:57:02.866685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 259/511] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:02 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:03.831307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:03.831916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:03.838910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:03.839422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:03.839850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:03.840813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:03.841258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:03.841675 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:03.878609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:03.881333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:03.888164 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:03.888661 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:03.889093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:03.889788 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:03.890242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:03.890663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:03.921233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-481ced5d-b6f8-4bb2-8cea-269eddaab726 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046525 Sep 11 00:57:03.921797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 260/512] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:03 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 11 00:57:03.988182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:04.002028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9dbf76e0-de27-4ad6-9d27-c4050170bf5e tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.174480 Sep 11 00:57:04.002598 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 253/513] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:03 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:04.004565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:04.005072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:04.011515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:04.011998 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:04.012432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:04.013151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:04.013606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:04.014028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:04.040424 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:57:04.122034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a2c487ff-99c9-4f43-9db6-67208ed70f5d tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 204 len: 0 microversion: 2.1 time: 0.121132 Sep 11 00:57:04.122814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 261/514] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:57:03 2022] DELETE /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 00:57:04.131908 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:04.132539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:04.138352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:04.138863 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:04.139295 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:04.139978 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:04.140424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:04.141032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:04.270039 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:04.277974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0d6844b2-84ba-4c78-89a9-2646ccce3ff9 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1688 microversion: 2.1 time: 0.150609 Sep 11 00:57:04.278537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 254/515] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:04 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1688 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:05.026613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:05.027344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:05.034033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:05.034519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:05.034992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:05.035684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:05.036140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:05.037245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:05.177722 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:05.186026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-792ecf0f-d8c3-4ac9-bc6d-e75cd6427101 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.163642 Sep 11 00:57:05.186592 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 262/516] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:05 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:05.301126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:05.301660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:05.308189 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:05.308674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:05.309107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:05.309791 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:05.310243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:05.310664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:05.440175 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:05.448625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7898434f-4b7c-4063-8e2e-93470371c589 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1688 microversion: 2.1 time: 0.151411 Sep 11 00:57:05.449186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 255/517] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:05 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1688 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:06.209602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:06.210243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:06.216763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:06.217243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:06.217674 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:06.218358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:06.218833 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:06.219258 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:06.359886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:06.368146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cf31d885-784e-4152-ba87-7a6f9f50d6a6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.162582 Sep 11 00:57:06.368713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 263/518] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:06 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:06.471604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:06.472228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:06.479347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:06.479841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:06.480279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:06.480976 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:06.481429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:06.481859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:06.611792 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:06.619586 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3b78046b-2697-40bf-9aea-98e8ae90d2ec tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1688 microversion: 2.1 time: 0.151869 Sep 11 00:57:06.620157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 256/519] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:06 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1688 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:07.390551 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:07.391125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:07.397513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:07.397996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:07.398430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:07.399159 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:07.399605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:07.400022 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:07.539994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:07.548472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c881b50c-f86d-4e48-9313-3a4f3da25abb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.161937 Sep 11 00:57:07.549036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 264/520] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:07 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:07.642642 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:07.643207 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:07.649782 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:07.650269 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:07.650696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:07.651401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:07.651875 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:07.652294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:07.778667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:07.787076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c6d63eed-34d5-4e0a-9aaa-c814b5eb0e38 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1688 microversion: 2.1 time: 0.147860 Sep 11 00:57:07.787633 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 257/521] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:07 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1688 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:08.573705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:08.574321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:08.580326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:08.580809 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:08.581243 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:08.581929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:08.582382 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:08.582826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:08.730327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:08.738764 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f0a868bd-a4af-4a07-a24b-2cf152775c4a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.169052 Sep 11 00:57:08.739347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 265/522] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:08 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:08.811216 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:08.811856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:08.819361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:08.820008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:08.820579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:08.821448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:08.822121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:08.822659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:08.960595 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:08.968316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-58da129a-9399-46dd-88a1-8c4b6499309c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1688 microversion: 2.1 time: 0.161735 Sep 11 00:57:08.968889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 258/523] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:08 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1688 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:09.427606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-48ea08a6-3b53-455e-9e99-f6f6b137ab5a req-866c8a5b-ae03-469f-b420-f58c48d18a64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb3cc25c-09a9-47c9-bc2e-d5227478ab70", "name": "network-vif-deleted", "tag": "f4ee5962-5a14-4994-83b5-3235d55d032f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:09.439201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-48ea08a6-3b53-455e-9e99-f6f6b137ab5a req-866c8a5b-ae03-469f-b420-f58c48d18a64 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:09.439937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-48ea08a6-3b53-455e-9e99-f6f6b137ab5a req-866c8a5b-ae03-469f-b420-f58c48d18a64 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:09.440297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-48ea08a6-3b53-455e-9e99-f6f6b137ab5a req-866c8a5b-ae03-469f-b420-f58c48d18a64 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:09.454251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-48ea08a6-3b53-455e-9e99-f6f6b137ab5a req-866c8a5b-ae03-469f-b420-f58c48d18a64 service nova] Creating event network-vif-deleted:f4ee5962-5a14-4994-83b5-3235d55d032f for instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 on hv-cinder-maste Sep 11 00:57:09.461636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-48ea08a6-3b53-455e-9e99-f6f6b137ab5a req-866c8a5b-ae03-469f-b420-f58c48d18a64 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037301 Sep 11 00:57:09.462245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 266/524] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:09.763775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:09.764592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:09.770546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:09.771503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:09.772170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:09.773810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:09.774268 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:09.774908 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:09.929969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:09.938180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-99c24b78-90ac-4f6a-8eaa-e9a50f0be78b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.178476 Sep 11 00:57:09.938753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 259/525] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:09 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:09.993665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:09.994363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:10.001000 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:10.001481 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:10.001914 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:10.002599 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:10.003095 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:10.003517 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:10.034819 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:10.035229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:10.035658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:10.036087 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lazy-loading 'fault' on Instance uuid eb3cc25c-09a9-47c9-bc2e-d5227478ab70 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:10.078097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:10.087358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-70870fdf-c2f9-4d55-8b47-46c9ad6df68f tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 200 len: 1334 microversion: 2.1 time: 0.098805 Sep 11 00:57:10.087928 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 267/526] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:09 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 1334 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:10.961823 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:10.962428 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:10.968994 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:10.969470 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:10.969896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:10.970571 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:10.971043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:10.971461 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.111473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:11.113334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:11.113943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: eb3cc25c-09a9-47c9-bc2e-d5227478ab70] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:11.120263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f965717a-41eb-4e81-948f-8e498ddde7a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.162677 Sep 11 00:57:11.120827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 260/527] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:10 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:11.121253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.121718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.122145 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.122931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.123379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.123797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.137888 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] HTTP exception thrown: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 could not be found. Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Request method failure captured: Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 HTTP/1.1 Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFZLkHjEmrxO7dxRA2K7NCkTkd6TbVWwXHlucfSpPoRvOHHB3oAJBaZcC7qeptnSWmQlNYC7VVp9rstzO1y1WIthC38SS1aTqw9V5bzg9Nair1V4TI7JOjqgjpvcNn4WH05oWOYYYTimbkYeWMgcOfVBndKoCvBDrkrzxnnHc0edIA Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:57:11.139585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_6a58aefc25e141ca93821a80859875e8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 5df6979effd84ece8d75b631ba7da196 Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 could not be found. Sep 11 00:57:11.141183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'eb3cc25c-09a9-47c9-bc2e-d5227478ab70'} Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 could not be found. Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:11.142017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 could not be found. Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 could not be found. Sep 11 00:57:11.143288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:11.144394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Returning 404 to user: Instance eb3cc25c-09a9-47c9-bc2e-d5227478ab70 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:11.144394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6af98e4a-845c-403d-ab92-5559a9ed8250 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70" status: 404 len: 111 microversion: 2.1 time: 0.031449 Sep 11 00:57:11.144394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 268/528] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:11 2022] GET /compute/v2.1/servers/eb3cc25c-09a9-47c9-bc2e-d5227478ab70 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 00:57:11.149438 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2b5aaafa-0606-496f-927d-a8aa690a6c11 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '.version_select at 0x7fc01aa7b040>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:11.167906 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2b5aaafa-0606-496f-927d-a8aa690a6c11 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-800080639" status: 202 len: 0 microversion: 2.1 time: 0.022286 Sep 11 00:57:11.168445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 261/529] 10.222.0.27 () {58 vars in 1218 bytes} [Sun Sep 11 00:57:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-800080639 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:57:11.249330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:11.249916 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:11.255986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.256471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.256900 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.257593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.258040 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.258458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.288419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d6685c1f-99b9-48f6-b8a7-0af593824d82 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments/3dc941c9-df60-4826-90b5-50998786adf4" status: 200 len: 194 microversion: 2.20 time: 0.043048 Sep 11 00:57:11.289006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 269/530] 10.222.0.27 () {60 vars in 1454 bytes} [Sun Sep 11 00:57:11 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments/3dc941c9-df60-4826-90b5-50998786adf4 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 11 00:57:11.370487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:11.371814 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:11.377776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.378332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.378859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.379655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.380210 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.381005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.464916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'services' on Instance uuid 7ed280f0-7beb-47b6-a543-0e3eb7083067 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:11.541419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-99868620-02dc-42e1-acff-f55c52e5e1bc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments/3dc941c9-df60-4826-90b5-50998786adf4" status: 202 len: 0 microversion: 2.20 time: 0.174577 Sep 11 00:57:11.542149 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 262/531] 10.222.0.27 () {60 vars in 1457 bytes} [Sun Sep 11 00:57:11 2022] DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments/3dc941c9-df60-4826-90b5-50998786adf4 => generated 0 bytes in 176 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:57:11.550650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:11.554454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:11.562471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.563185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.564180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.565708 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:11.566797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:11.567330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:11.597254 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:11.604209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-85ac0a1c-cf06-4671-92a2-dbb36e485513 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057529 Sep 11 00:57:11.604773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 270/532] 10.222.0.27 () {60 vars in 1343 bytes} [Sun Sep 11 00:57:11 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:57:12.145208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:12.145910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:12.152347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.152891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.153333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.154025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.154480 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.154945 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.306560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:12.314882 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-15484c3f-804a-435a-9d8f-954f8650d8a2 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.174485 Sep 11 00:57:12.315458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 263/533] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:12 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:12.618012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:12.620598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:12.628576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.629060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.629492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.630187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.630638 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.631090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.664894 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4f27d1ee-8597-4ff7-b573-7ca86bb4559c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.051760 Sep 11 00:57:12.665448 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 271/534] 10.222.0.27 () {60 vars in 1343 bytes} [Sun Sep 11 00:57:12 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:12.728734 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:12.729235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:12.735183 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.735672 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.736114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.736802 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.737260 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.737687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.762733 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:57:12.807626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:12.839176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c39354e2-a18b-4473-ba22-54d80412f813 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 204 len: 0 microversion: 2.20 time: 0.114029 Sep 11 00:57:12.839668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 264/535] 10.222.0.27 () {60 vars in 1280 bytes} [Sun Sep 11 00:57:12 2022] DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 11 00:57:12.847859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:12.848467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:12.854618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.855147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.855581 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.856265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:12.856713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:12.857131 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:12.989575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:12.998062 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:12.999514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:13.000662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8e625e7a-a49f-4e1b-9999-8eb25ee2b133 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.156651 Sep 11 00:57:13.001208 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 272/536] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:12 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:13.338582 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:13.339326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:13.345702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:13.346129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:13.346580 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:13.347301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:13.347756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:13.348181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:13.500321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:13.509834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3176860d-681b-43b7-803d-8b603d875a59 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.175451 Sep 11 00:57:13.510396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 265/537] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:13 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:14.026491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:14.027196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:14.034016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.034492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.034953 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.035637 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.036083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.036499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.166416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:14.174402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:14.175903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:14.177047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4e33b220-0248-4d87-a181-b6d388a1b628 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.155415 Sep 11 00:57:14.177595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 273/538] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:14 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:14.220999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:14.221501 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:14.227643 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.228142 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.228558 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.229244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.229694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.230117 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.253973 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:57:14.329419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-532ae9f8-4e52-480e-9782-183866e43c81 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "DELETE /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 204 len: 0 microversion: 2.1 time: 0.111881 Sep 11 00:57:14.329913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 266/539] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:57:14 2022] DELETE /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:57:14.338350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:14.338998 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:14.344799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.345275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.345703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.346381 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.346854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.347274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.478586 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:14.486341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-54c1983e-8202-4fbb-bf44-94753715a42c tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1508 microversion: 2.1 time: 0.152179 Sep 11 00:57:14.486933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 274/540] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:14 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1508 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:14.533632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:14.534363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:14.541515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.541999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.542432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.543180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:14.543626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:14.544050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:14.684232 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:14.692333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9e49b9e6-dd97-4aeb-848c-77a3312570ea tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.163156 Sep 11 00:57:14.692891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 267/541] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:14 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:15.204316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:15.204953 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:15.211945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:15.212427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:15.212854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:15.213572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:15.214021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:15.214441 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:15.343452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:15.351570 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:15.353011 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:15.354217 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-55a9da8b-27e2-45a1-99f7-92318486dc14 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.153833 Sep 11 00:57:15.354724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 275/542] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:15 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:15.510371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:15.511009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:15.517202 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:15.517692 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:15.518134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:15.518856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:15.519316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:15.519743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:15.645071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:15.653243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5b4cde32-70fb-446a-a4a4-992876704329 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1508 microversion: 2.1 time: 0.147299 Sep 11 00:57:15.653805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 268/543] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:15 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1508 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:15.715850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:15.716820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:15.722639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:15.723153 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:15.723581 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:15.724267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:15.724781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:15.725219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:15.866099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:15.874326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c4459ea7-7ed7-4220-9d63-1245bc223daa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.162269 Sep 11 00:57:15.874923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 276/544] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:15 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:16.379891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:16.380421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:16.387064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:16.387542 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:16.387972 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:16.388682 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:16.389128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:16.389545 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:16.514491 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:16.522496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:16.523962 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:16.525108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c52ed572-fec4-4468-9df6-deba8f0d1a5b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.149295 Sep 11 00:57:16.525652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 269/545] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:16 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:16.676955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:16.677505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:16.684634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:16.685119 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:16.685552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:16.686239 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:16.686691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:16.687144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:16.813621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:16.821545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cf8f2355-abe0-44ea-b603-3990c6abedb1 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1508 microversion: 2.1 time: 0.148556 Sep 11 00:57:16.822107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 277/546] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:16 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1508 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:16.897795 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:16.898411 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:16.904868 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:16.905352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:16.905784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:16.906473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:16.906955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:16.907424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:17.050913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:17.059057 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7e0267d0-e3b8-471e-b707-52c2ede1edc6 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1755 microversion: 2.1 time: 0.165559 Sep 11 00:57:17.059618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 270/547] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:16 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1755 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:17.551305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:17.551913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:17.558414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:17.559023 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:17.559605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:17.560912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:17.561468 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:17.561980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:17.689091 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:17.697788 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:17.699206 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:17.700354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-97affaa9-2ced-4b6e-9daa-793b241e69c5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.153214 Sep 11 00:57:17.700925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 278/548] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:17 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:17.844976 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:17.845620 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:17.851975 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:17.852430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:17.852872 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:17.853573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:17.854416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:17.854673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:17.986227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:17.994711 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-09e6f9d6-229e-4efc-a988-b37ee0780fa8 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1508 microversion: 2.1 time: 0.153577 Sep 11 00:57:17.995337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 271/549] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:17 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1508 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:18.081637 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:18.082271 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:18.089886 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:18.090394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:18.090844 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:18.091536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:18.091986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:18.092400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:18.239171 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:18.247452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-12cfb661-d865-402f-8e4d-4104d7bd26fb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1758 microversion: 2.1 time: 0.169699 Sep 11 00:57:18.248022 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 279/550] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:18 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1758 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:18.727424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:18.728042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:18.733867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:18.734347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:18.734779 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:18.735488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:18.735940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:18.736356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:18.873411 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:18.884060 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:18.886427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:18.888011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ca62d259-a5a9-41bd-9958-a207438cea0e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1722 microversion: 2.20 time: 0.166128 Sep 11 00:57:18.889132 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 272/551] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:18 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1722 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:57:18.971743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-f02d3ef3-b6d2-426c-8baa-d82ab1e3047e req-57109691-3844-4d37-9bf2-c36226fcc7d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62f2702d-714f-47f4-87e5-25f0283ee9a5", "name": "network-vif-deleted", "tag": "d093dbf0-edb8-4356-a6c6-97c0aa32850c"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:19.009313 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f02d3ef3-b6d2-426c-8baa-d82ab1e3047e req-57109691-3844-4d37-9bf2-c36226fcc7d1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:19.015483 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f02d3ef3-b6d2-426c-8baa-d82ab1e3047e req-57109691-3844-4d37-9bf2-c36226fcc7d1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:19.016031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f02d3ef3-b6d2-426c-8baa-d82ab1e3047e req-57109691-3844-4d37-9bf2-c36226fcc7d1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:19.018553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:19.019758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:19.032347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-f02d3ef3-b6d2-426c-8baa-d82ab1e3047e req-57109691-3844-4d37-9bf2-c36226fcc7d1 service nova] Creating event network-vif-deleted:d093dbf0-edb8-4356-a6c6-97c0aa32850c for instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 on hv-cinder-maste Sep 11 00:57:19.034086 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:19.035029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:19.038773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:19.039828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:19.040318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:19.040949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:19.041420 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-f02d3ef3-b6d2-426c-8baa-d82ab1e3047e req-57109691-3844-4d37-9bf2-c36226fcc7d1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079137 Sep 11 00:57:19.045677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 280/552] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:19.128553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:19.136844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-de75da46-6e89-4837-b14c-bf9a23b61414 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 200 len: 1292 microversion: 2.1 time: 0.122964 Sep 11 00:57:19.137414 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 273/553] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:19 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 1292 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:19.918103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:19.918827 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:19.925334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:19.925888 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:19.926416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:19.927390 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:19.928575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:19.929287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:19.963250 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:19.964302 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:19.964973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:19.965429 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'fault' on Instance uuid 7ed280f0-7beb-47b6-a543-0e3eb7083067 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:20.012173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:20.020610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:20.022695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:20.024215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-656c191c-6603-42ef-8a36-147ef2afcc6a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1371 microversion: 2.20 time: 0.110236 Sep 11 00:57:20.024800 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 281/554] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:19 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1371 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:20.160204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:20.161384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 62f2702d-714f-47f4-87e5-25f0283ee9a5] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:20.169424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:20.169916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:20.170358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:20.171139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:20.171599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:20.172027 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:20.188448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] HTTP exception thrown: Instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 could not be found. Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Request method failure captured: Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 HTTP/1.1 Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFZLkHjEmrxO7dxRA2K7NCkTkd6TbVWwXHlucfSpPoRvOHHB3oAJBaZcC7qeptnSWmQlNYC7VVp9rstzO1y1WIthC38SS1aTqw9V5bzg9Nair1V4TI7JOjqgjpvcNn4WH05oWOYYYTimbkYeWMgcOfVBndKoCvBDrkrzxnnHc0edIA Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:57:20.190127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_6a58aefc25e141ca93821a80859875e8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 5df6979effd84ece8d75b631ba7da196 Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 could not be found. Sep 11 00:57:20.191630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '62f2702d-714f-47f4-87e5-25f0283ee9a5'} Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 could not be found. Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:20.192396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 could not be found. Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 could not be found. Sep 11 00:57:20.193683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:20.195095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Returning 404 to user: Instance 62f2702d-714f-47f4-87e5-25f0283ee9a5 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:20.195095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8d3dfa8f-78f1-48bc-9c6e-b62c6a339539 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5" status: 404 len: 111 microversion: 2.1 time: 0.035572 Sep 11 00:57:20.195095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 274/555] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:20 2022] GET /compute/v2.1/servers/62f2702d-714f-47f4-87e5-25f0283ee9a5 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:57:20.200783 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:20.201284 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:20.207320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:20.207801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:20.208235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:20.209101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:20.209559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:20.209982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:20.235405 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:57:20.316694 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d0b983c5-6b90-4067-8293-8e6f0e9ddb49 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "DELETE /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 204 len: 0 microversion: 2.1 time: 0.119246 Sep 11 00:57:20.317743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 282/556] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:57:20 2022] DELETE /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:57:20.326469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:20.327405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:20.334138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:20.334721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:20.335384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:20.336082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:20.336537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:20.336964 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:20.464405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:20.472456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d89b1fee-d660-4fda-9086-ce935bcaac15 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.150223 Sep 11 00:57:20.473009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 275/557] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:20 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:21.050433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:21.051072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:21.057508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:21.057991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:21.058431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:21.059144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:21.059615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:21.060017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:21.073442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-6bf6a76d-7980-45d8-b474-15311a181dc6 req-b4b95376-82a5-4ffc-922b-9eaa17869d62 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ed280f0-7beb-47b6-a543-0e3eb7083067", "name": "network-vif-deleted", "tag": "8a166d16-f941-44a7-ba58-ccb7e2c62c18"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:21.083128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6bf6a76d-7980-45d8-b474-15311a181dc6 req-b4b95376-82a5-4ffc-922b-9eaa17869d62 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:21.083607 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6bf6a76d-7980-45d8-b474-15311a181dc6 req-b4b95376-82a5-4ffc-922b-9eaa17869d62 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:21.084751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6bf6a76d-7980-45d8-b474-15311a181dc6 req-b4b95376-82a5-4ffc-922b-9eaa17869d62 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:21.092601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:21.093090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:21.093519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:21.093964 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'fault' on Instance uuid 7ed280f0-7beb-47b6-a543-0e3eb7083067 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:21.096169 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-6bf6a76d-7980-45d8-b474-15311a181dc6 req-b4b95376-82a5-4ffc-922b-9eaa17869d62 service nova] Creating event network-vif-deleted:8a166d16-f941-44a7-ba58-ccb7e2c62c18 for instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 on hv-cinder-maste Sep 11 00:57:21.101274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-6bf6a76d-7980-45d8-b474-15311a181dc6 req-b4b95376-82a5-4ffc-922b-9eaa17869d62 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032644 Sep 11 00:57:21.102035 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 276/558] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:57:21.138699 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:21.147668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:21.149108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:21.150244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-661e958f-090b-4edf-b7f3-8e3c18be8f7e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1371 microversion: 2.20 time: 0.104262 Sep 11 00:57:21.150816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 283/559] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:21 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1371 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:21.497518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:21.498221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:21.504379 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:21.504861 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:21.505298 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:21.506610 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:21.507087 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:21.507516 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:21.633923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:21.641552 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d8d93f9a-431f-4cb4-94f8-2a282ff51a96 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.148846 Sep 11 00:57:21.642114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 277/560] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:21 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:22.175025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:22.175634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:22.182045 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:22.182520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:22.182977 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:22.183662 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:22.184106 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:22.184523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:22.215289 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:22.215762 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:22.216185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:22.216619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'fault' on Instance uuid 7ed280f0-7beb-47b6-a543-0e3eb7083067 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:22.255974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:22.263927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:22.265356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:22.266497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a09fd27d-22d4-42b8-90c9-9246152aa97d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 200 len: 1371 microversion: 2.20 time: 0.095374 Sep 11 00:57:22.267077 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 284/561] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:22 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 1371 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:22.664847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:22.665391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:22.671848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:22.672335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:22.672772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:22.673461 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:22.673915 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:22.674342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:22.803044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:22.810761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0fa7ee3b-dc1b-47eb-ae85-920dde97e56a tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.149783 Sep 11 00:57:22.811358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 278/562] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:22 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:23.292946 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:23.293570 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:23.299977 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:23.300453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:23.300876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:23.301558 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:23.301997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:23.303099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:23.316622 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] HTTP exception thrown: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Request method failure captured: Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 HTTP/1.1 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFh7LqKP94hM4-KTW8o175jD3zotXAsDceOYAGcfUA4X4HFr7u0MTuULetTgdHMY9LAcSb8D8pljyJ-YTXo3ITsO2j7rVAonZoyAn6wNKepm-7hai-7Hfpk9sfmJyx8xRg1ZHt0EPAu5pez6rZ4rAig9k3SUOLp1byI1JCSrGdbVH8 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Openstack-Nova-Api-Version: 2.20 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:57:23.318397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 73a3d3c48913474ab8fa8fedd1bc7d03 Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 00:57:23.320185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '7ed280f0-7beb-47b6-a543-0e3eb7083067'} Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:23.321092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 00:57:23.322616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:23.332687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Returning 404 to user: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:23.332687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-74e0ee58-2c52-4923-a043-0c93c41a215b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 404 len: 111 microversion: 2.20 time: 0.031175 Sep 11 00:57:23.332687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 285/563] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:23 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 11 00:57:23.333036 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-56f9b5b7-146f-4963-8ed1-4af57fe95371 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '.version_select at 0x7fc01aa33ee0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:23.348191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-56f9b5b7-146f-4963-8ed1-4af57fe95371 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1743888382" status: 204 len: 0 microversion: 2.20 time: 0.023041 Sep 11 00:57:23.348793 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 279/564] 10.222.0.27 () {60 vars in 1262 bytes} [Sun Sep 11 00:57:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1743888382 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 11 00:57:23.835654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:23.836269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:23.842407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:23.842951 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:23.843356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:23.844053 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:23.844723 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:23.845151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:23.849978 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:24.073709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5df6979effd84ece8d75b631ba7da196', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a58aefc25e141ca93821a80859875e8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:24.093167 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-52f15771-4db4-4b2e-96a8-024b79f043eb req-1eadb7d9-856a-48e6-9903-9e0cb82dd5d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9acff3d4-2dd1-4809-bc64-2dfc7829bfc6", "name": "network-vif-deleted", "tag": "1fc55f70-880d-4007-99be-49ec52f1b61e"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:24.096880 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-de913d81-37c3-4100-b485-5cb7b7960751 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 200 len: 1522 microversion: 2.1 time: 0.266015 Sep 11 00:57:24.098775 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 286/565] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:23 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 1522 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:24.104400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ff25b351-47e6-4ced-95f9-d83d74d94552 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'create', calling method: .version_select at 0x7fc01acfbca0>, body: {"keypair": {"name": "tempest-keypair-1815041047", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGkq3yrk+5nK4IKTCYRKu8UMw0++Tw3bgcuIzSn6PKN74GFsYktGQ5dWXqs/6ZypMK6k69GYQVufY5YTRRIPEkszt/Zuipeg4TRKoqeV3MxhBv7A/+Jjbpui3tkrD25TNQ=="}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:24.124324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-52f15771-4db4-4b2e-96a8-024b79f043eb req-1eadb7d9-856a-48e6-9903-9e0cb82dd5d8 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:24.124923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-52f15771-4db4-4b2e-96a8-024b79f043eb req-1eadb7d9-856a-48e6-9903-9e0cb82dd5d8 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:24.125442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-52f15771-4db4-4b2e-96a8-024b79f043eb req-1eadb7d9-856a-48e6-9903-9e0cb82dd5d8 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:24.137502 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-52f15771-4db4-4b2e-96a8-024b79f043eb req-1eadb7d9-856a-48e6-9903-9e0cb82dd5d8 service nova] Creating event network-vif-deleted:1fc55f70-880d-4007-99be-49ec52f1b61e for instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 on hv-cinder-maste Sep 11 00:57:24.137684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-ff25b351-47e6-4ced-95f9-d83d74d94552 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for project fd9221b2e81a46f69864e02ec4d42672. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:57:24.142071 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-ff25b351-47e6-4ced-95f9-d83d74d94552 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for user 73a3d3c48913474ab8fa8fedd1bc7d03 and project fd9221b2e81a46f69864e02ec4d42672. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:57:24.142238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-52f15771-4db4-4b2e-96a8-024b79f043eb req-1eadb7d9-856a-48e6-9903-9e0cb82dd5d8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055152 Sep 11 00:57:24.142855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 280/566] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:24.167095 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ff25b351-47e6-4ced-95f9-d83d74d94552 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.066358 Sep 11 00:57:24.167681 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 287/567] 10.222.0.27 () {62 vars in 1200 bytes} [Sun Sep 11 00:57:24 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 68 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 11 00:57:25.130720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:25.131656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] [instance: 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:25.137678 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:25.138157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:25.138598 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:25.139378 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:25.139835 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:25.140256 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:25.154517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] HTTP exception thrown: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 could not be found. Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Request method failure captured: Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 HTTP/1.1 Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFZLkHjEmrxO7dxRA2K7NCkTkd6TbVWwXHlucfSpPoRvOHHB3oAJBaZcC7qeptnSWmQlNYC7VVp9rstzO1y1WIthC38SS1aTqw9V5bzg9Nair1V4TI7JOjqgjpvcNn4WH05oWOYYYTimbkYeWMgcOfVBndKoCvBDrkrzxnnHc0edIA Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:57:25.156205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/6a58aefc25e141ca93821a80859875e8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_6a58aefc25e141ca93821a80859875e8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 6a58aefc25e141ca93821a80859875e8 Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-2044416371 Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 5df6979effd84ece8d75b631ba7da196 Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-ServerRescueNegativeTestJSON-2044416371-project Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 could not be found. Sep 11 00:57:25.157813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '9acff3d4-2dd1-4809-bc64-2dfc7829bfc6'} Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 could not be found. Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:25.158949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 could not be found. Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 could not be found. Sep 11 00:57:25.160359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:25.162289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] Returning 404 to user: Instance 9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:25.162289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e777bf44-84f2-40dd-a14e-23354f55b123 tempest-ServerRescueNegativeTestJSON-2044416371 tempest-ServerRescueNegativeTestJSON-2044416371-project] 10.222.0.27 "GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6" status: 404 len: 111 microversion: 2.1 time: 0.031814 Sep 11 00:57:25.162289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 281/568] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:25 2022] GET /compute/v2.1/servers/9acff3d4-2dd1-4809-bc64-2dfc7829bfc6 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:57:25.662460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1823321323", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3c897342-0f04-45f8-a9ef-0bc52e6c6ab5"}], "security_groups": [{"name": "tempest-securitygroup--1996936808"}], "key_name": "tempest-keypair-1815041047", "user_data": "}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:25.770531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] validate_networks() for [('3c897342-0f04-45f8-a9ef-0bc52e6c6ab5', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:57:26.355793 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:57:26.356297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:57:26.356851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:57:26.357258 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:57:26.357666 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:57:26.360641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:26.368352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:26.368832 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:26.369273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:26.370000 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:26.370451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:26.370900 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:26.388960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for project fd9221b2e81a46f69864e02ec4d42672. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:57:26.393016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Getting quotas for user 73a3d3c48913474ab8fa8fedd1bc7d03 and project fd9221b2e81a46f69864e02ec4d42672. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:57:26.404810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:57:26.407120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:57:26.458981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-74ac11db-029e-4666-b4a2-535bad145c3c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.801249 Sep 11 00:57:26.459805 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 288/569] 10.222.0.27 () {62 vars in 1188 bytes} [Sun Sep 11 00:57:25 2022] POST /compute/v2.1/servers => generated 402 bytes in 803 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 11 00:57:26.470950 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6976ed10-1961-4b0d-9299-d45ef45d1542 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:26.471698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6976ed10-1961-4b0d-9299-d45ef45d1542 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:26.529378 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6976ed10-1961-4b0d-9299-d45ef45d1542 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:26.540145 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6976ed10-1961-4b0d-9299-d45ef45d1542 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:26.541723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6976ed10-1961-4b0d-9299-d45ef45d1542 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:26.543009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6976ed10-1961-4b0d-9299-d45ef45d1542 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1282 microversion: 2.20 time: 0.075919 Sep 11 00:57:26.543625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 282/570] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:26 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1282 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:27.570098 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:27.571386 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:27.577419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:27.578079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:27.578593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:27.579302 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:27.579982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:27.580549 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:27.654902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:27.664186 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:27.665852 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:27.667020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c31774ad-017b-47e9-b205-5c9c6626cd4d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1334 microversion: 2.20 time: 0.101969 Sep 11 00:57:27.667960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 289/571] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:27 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1334 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:28.695047 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:28.695684 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:28.706369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:28.709499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:28.709955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:28.712007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:28.713816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:28.714274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:29.020216 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:29.061115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:29.062711 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:29.063865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c611d95a-cc36-4651-9e0d-af817cf3c241 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.375121 Sep 11 00:57:29.075253 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 283/572] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:28 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:29.591278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-779353aa-faf8-4d73-b1a7-6f8bd1c2c32f req-7866beaa-3185-4957-833e-ade0bd027937 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:29.602879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-779353aa-faf8-4d73-b1a7-6f8bd1c2c32f req-7866beaa-3185-4957-833e-ade0bd027937 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:29.603351 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-779353aa-faf8-4d73-b1a7-6f8bd1c2c32f req-7866beaa-3185-4957-833e-ade0bd027937 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:29.603944 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-779353aa-faf8-4d73-b1a7-6f8bd1c2c32f req-7866beaa-3185-4957-833e-ade0bd027937 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:29.629945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-779353aa-faf8-4d73-b1a7-6f8bd1c2c32f req-7866beaa-3185-4957-833e-ade0bd027937 service nova] Creating event network-changed:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 00:57:29.639706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-779353aa-faf8-4d73-b1a7-6f8bd1c2c32f req-7866beaa-3185-4957-833e-ade0bd027937 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052631 Sep 11 00:57:29.641321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 290/573] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:30.090217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:30.091056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:30.097661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:30.098158 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:30.098606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:30.099863 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:30.100321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:30.100752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:30.124718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:30.263188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:30.272188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:30.273920 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:30.275419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-eb26fe4a-90af-443f-bc92-1d1312f20c1d tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.189443 Sep 11 00:57:30.276333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 284/574] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:30 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:30.974302 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:30.975167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:30.981796 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:30.982275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:30.982699 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:30.983454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:30.983913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:30.984487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:31.011694 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1aff61c9-f7c6-419f-a863-2b6ce4071791 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/ips" status: 200 len: 150 microversion: 2.1 time: 0.042250 Sep 11 00:57:31.012259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 291/575] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:57:30 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/ips => generated 150 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:31.301432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:31.303224 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:31.310850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=85,nova_api:DELETE=2,nova_api:UPDATE=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:31.313030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:31.313537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:31.314178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:31.315351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:31.315815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:31.316304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:31.466009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:31.475085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:31.476979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:31.478100 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7129ad17-5840-4f72-8c38-4712ba659e24 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.180817 Sep 11 00:57:31.478659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 285/576] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:31 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:57:32.425918 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "06d7b23b-f453-4b93-b690-8296e9920d33"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:32.430395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:32.449264 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:32.450018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:32.450731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:32.451630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:32.452378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:32.452955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:32.507664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:32.508285 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:32.515955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:32.518896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:32.519354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:32.520076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:32.520539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:32.521522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:32.673742 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:32.682688 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:32.684176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:32.685356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-385c977e-a657-4715-8929-edb867c779af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.181770 Sep 11 00:57:32.686079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 286/577] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:32 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:57:32.756285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9f4e42c0-6e0f-4890-afe4-5bd3419f6211 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.333184 Sep 11 00:57:32.756875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 292/578] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:57:32 2022] POST /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments => generated 194 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:33.712865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:33.718533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:33.725434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:33.725927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:33.726574 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:33.727335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:33.727779 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:33.728345 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:33.870964 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:33.880280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:33.883956 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:33.885329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a291e589-aaf7-4440-b661-a86704ad8d40 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.176921 Sep 11 00:57:33.886332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 287/579] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:33 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:33.964815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-1f437da8-c27d-4a73-b645-fe6976b0af8b req-a7039b42-d9f8-40ff-9cba-fd6be0ba6338 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "name": "network-vif-plugged", "status": "completed", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:33.975533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-1f437da8-c27d-4a73-b645-fe6976b0af8b req-a7039b42-d9f8-40ff-9cba-fd6be0ba6338 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:33.976813 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-1f437da8-c27d-4a73-b645-fe6976b0af8b req-a7039b42-d9f8-40ff-9cba-fd6be0ba6338 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:33.977237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-1f437da8-c27d-4a73-b645-fe6976b0af8b req-a7039b42-d9f8-40ff-9cba-fd6be0ba6338 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:33.988535 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-1f437da8-c27d-4a73-b645-fe6976b0af8b req-a7039b42-d9f8-40ff-9cba-fd6be0ba6338 service nova] Creating event network-vif-plugged:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 00:57:33.994333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-1f437da8-c27d-4a73-b645-fe6976b0af8b req-a7039b42-d9f8-40ff-9cba-fd6be0ba6338 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034219 Sep 11 00:57:33.995319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 293/580] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:34.910146 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:34.910894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:34.917650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:34.918132 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:34.918567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:34.919278 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:34.919726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:34.920144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:35.051780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:35.060201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:35.061663 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:35.062758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-090dece0-eb99-4b46-abd8-d7e4927549dd tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.156806 Sep 11 00:57:35.063326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 288/581] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:34 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:36.089459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:36.090230 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:36.096382 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:36.096861 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:36.097292 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:36.097978 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:36.098430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:36.098883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:36.377551 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:36.379864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:36.387597 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:36.389030 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:36.390135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-adf19e19-8767-4d4e-a152-8e25abed0b90 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.306067 Sep 11 00:57:36.390690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 294/582] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:36 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:37.414989 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:37.415600 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:37.417982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:37.423790 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:37.424245 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:37.424676 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:37.425362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:37.425859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:37.426277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:37.619647 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:37.634325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:37.635868 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:37.637334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6128d380-7044-4414-b41d-319a2e4074b5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1408 microversion: 2.20 time: 0.226130 Sep 11 00:57:37.637960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 289/583] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:37 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1408 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:38.664376 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:38.665113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:38.668003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=10 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:38.673774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:38.674255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:38.675434 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:38.676135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:38.676584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:38.677182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:38.811043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:38.819267 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:38.821869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:38.823060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-22bb668c-4ae2-4322-889d-f8470b6fcf1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1599 microversion: 2.20 time: 0.163350 Sep 11 00:57:38.823601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 295/584] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:57:38 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:57:38.847221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:38.847720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:38.855013 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:38.855506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:38.855952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:38.856646 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:38.857105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:38.857532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:38.939895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1d15afbc-046a-47ff-9c65-6ff407c5de39 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.096539 Sep 11 00:57:38.940510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 290/585] 10.222.0.27 () {60 vars in 1316 bytes} [Sun Sep 11 00:57:38 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:57:40.128291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-142036c3-d555-4187-b225-755620bfe668 req-c2844782-fd2c-42b2-abe2-e605476bc474 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:40.138559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-142036c3-d555-4187-b225-755620bfe668 req-c2844782-fd2c-42b2-abe2-e605476bc474 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.139295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-142036c3-d555-4187-b225-755620bfe668 req-c2844782-fd2c-42b2-abe2-e605476bc474 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.139733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-142036c3-d555-4187-b225-755620bfe668 req-c2844782-fd2c-42b2-abe2-e605476bc474 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.151935 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-142036c3-d555-4187-b225-755620bfe668 req-c2844782-fd2c-42b2-abe2-e605476bc474 service nova] Creating event network-changed:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 00:57:40.157481 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-142036c3-d555-4187-b225-755620bfe668 req-c2844782-fd2c-42b2-abe2-e605476bc474 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032848 Sep 11 00:57:40.158520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 296/586] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:40.397099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "06d7b23b-f453-4b93-b690-8296e9920d33"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:40.400698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:40.408912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.409737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.410481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.411743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.412333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.413560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.513761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Invalid volume: volume 06d7b23b-f453-4b93-b690-8296e9920d33 is already attached to instances: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: POST /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments HTTP/1.1 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 74 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:57:40.515450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {"volumeAttachment": {"volumeId": "06d7b23b-f453-4b93-b690-8296e9920d33"}} Sep 11 00:57:40.517228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Invalid volume: volume 06d7b23b-f453-4b93-b690-8296e9920d33 is already attached to instances: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'server_id': 'b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650', 'body': {'volumeAttachment': {'volumeId': '06d7b23b-f453-4b93-b690-8296e9920d33'}}} Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 06d7b23b-f453-4b93-b690-8296e9920d33 is already attached to instances: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:40.518111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 06d7b23b-f453-4b93-b690-8296e9920d33 is already attached to instances: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 06d7b23b-f453-4b93-b690-8296e9920d33 is already attached to instances: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 400 to user: Invalid volume: volume 06d7b23b-f453-4b93-b690-8296e9920d33 is already attached to instances: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-91f96cfd-8c71-49fa-b779-5c1bd16fb8e7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.126208 Sep 11 00:57:40.521918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 291/587] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:57:40 2022] POST /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments => generated 174 bytes in 127 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 11 00:57:40.594493 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:40.595045 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:40.602209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.602826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.603344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.604115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.604644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.605147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.690973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'services' on Instance uuid b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:40.779830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1a876cb3-12dd-4cf6-a2ce-d7c0e492b1a6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments/06d7b23b-f453-4b93-b690-8296e9920d33" status: 202 len: 0 microversion: 2.1 time: 0.189481 Sep 11 00:57:40.781110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 297/588] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:57:40 2022] DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments/06d7b23b-f453-4b93-b690-8296e9920d33 => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:57:40.791396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:40.793680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:40.800210 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.801046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.801709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.802634 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:40.803134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:40.803569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:40.839886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-097468ad-3306-46f9-841d-f13ecf6955df tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054163 Sep 11 00:57:40.844989 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 292/589] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:40 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:41.854413 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:41.856743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:41.862882 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:41.863360 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:41.863784 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:41.864460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:41.864903 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:41.865320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:41.895275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e94f33bf-eb79-4733-bd97-fc516fe619ee tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046373 Sep 11 00:57:41.895855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 298/590] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:41 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 00:57:41.963415 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:41.963979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:41.970593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:41.971226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:41.971732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:41.972532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:41.973095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:41.973519 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:41.999548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:57:42.081173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cf264b46-6603-4025-ad15-9a13753ceb6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 204 len: 0 microversion: 2.1 time: 0.122218 Sep 11 00:57:42.081777 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 293/591] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:57:41 2022] DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:57:42.091118 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:42.091729 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:42.099108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:42.099812 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:42.101025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:42.102067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:42.102724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:42.103363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:42.236941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:42.245334 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ee144c5c-a754-40d9-9fc3-b8c92ed20326 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1682 microversion: 2.1 time: 0.158470 Sep 11 00:57:42.245900 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 299/592] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:42 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1682 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:43.269860 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:43.270561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:43.277517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:43.278001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:43.278436 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:43.279148 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:43.279602 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:43.280025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:43.409997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:43.417698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e5dee234-1e7c-48c8-a78b-f730c3f1fd6e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1682 microversion: 2.1 time: 0.152082 Sep 11 00:57:43.418256 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 294/593] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:43 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1682 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:44.447156 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:44.447156 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:44.451221 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:44.451831 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:44.452309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:44.453070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:44.453523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:44.453947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:44.583872 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:44.591909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-daa219a5-75f3-4354-8ac6-1a43ba7b877d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1682 microversion: 2.1 time: 0.154425 Sep 11 00:57:44.592490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 300/594] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:44 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1682 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:45.126499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-00b6d3fc-f30a-4e4f-94bb-3bebd887bca0 req-dfc0c939-b5d1-488c-8ca8-9d1a416b3e3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650", "name": "network-vif-deleted", "tag": "6a7c8c37-25f4-480e-a003-6a8f1f5f186d"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:45.140127 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-00b6d3fc-f30a-4e4f-94bb-3bebd887bca0 req-dfc0c939-b5d1-488c-8ca8-9d1a416b3e3e service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:45.140707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-00b6d3fc-f30a-4e4f-94bb-3bebd887bca0 req-dfc0c939-b5d1-488c-8ca8-9d1a416b3e3e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:45.141529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-00b6d3fc-f30a-4e4f-94bb-3bebd887bca0 req-dfc0c939-b5d1-488c-8ca8-9d1a416b3e3e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:45.160346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-00b6d3fc-f30a-4e4f-94bb-3bebd887bca0 req-dfc0c939-b5d1-488c-8ca8-9d1a416b3e3e service nova] Creating event network-vif-deleted:6a7c8c37-25f4-480e-a003-6a8f1f5f186d for instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 on hv-cinder-maste Sep 11 00:57:45.167498 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-00b6d3fc-f30a-4e4f-94bb-3bebd887bca0 req-dfc0c939-b5d1-488c-8ca8-9d1a416b3e3e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045249 Sep 11 00:57:45.168715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 295/595] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:45.615029 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:45.615955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:45.622960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:45.623524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:45.624027 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:45.624795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:45.625320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:45.625848 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:45.658269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:45.658763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:45.659218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:45.659665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'fault' on Instance uuid b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:45.701828 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:45.711314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d427ee8d-5f46-4b52-9e73-0fdc0d9eb653 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 200 len: 1331 microversion: 2.1 time: 0.100015 Sep 11 00:57:45.712337 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 301/596] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:45 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 1331 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:46.737043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:46.737755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:46.745337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:46.745827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:46.746261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:46.746988 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:46.747459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:46.747871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:46.761575 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 HTTP/1.1 Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:57:46.763242 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:57:46.764786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650'} Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:46.765566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:57:46.766879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:57:46.768108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:46.768108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b3fa68ff-93ea-4b48-82bf-a9370ce0b307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 404 len: 111 microversion: 2.1 time: 0.031816 Sep 11 00:57:46.768108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 296/597] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:46 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:57:46.773589 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-938b350e-3750-4a97-8233-e5cc4db1fd7e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '.version_select at 0x7fc01af599d0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:46.793076 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-938b350e-3750-4a97-8233-e5cc4db1fd7e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1494663280" status: 202 len: 0 microversion: 2.1 time: 0.023190 Sep 11 00:57:46.793892 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 302/598] 10.222.0.27 () {58 vars in 1221 bytes} [Sun Sep 11 00:57:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1494663280 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:57:47.422499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d19882f2-9fdc-4d9a-bb8c-a7ad6f31b64e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: .version_select at 0x7fc01aa8bd30>, body: {"keypair": {"name": "tempest-keypair-308857369", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5pR7gSffMt/zA9kYLSK4zkwq44XelK8OvyDA3cjw4tHd3mqWNSBob9I8LHtxURXGmF45qrcFb5cae21APV655vbYeJaBQS1ohRwnIR1Xd21w3uUkW9uLWy7RIf4VI5FQ=="}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:47.438218 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-d19882f2-9fdc-4d9a-bb8c-a7ad6f31b64e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:57:47.442042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-d19882f2-9fdc-4d9a-bb8c-a7ad6f31b64e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:57:47.463996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d19882f2-9fdc-4d9a-bb8c-a7ad6f31b64e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045393 Sep 11 00:57:47.464576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 297/599] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 00:57:47 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:48.729653 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-855769917", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e5ef326a-1def-4c9c-ad55-20a6d9fdbde1"}], "security_groups": [{"name": "tempest-securitygroup--940293107"}], "key_name": "tempest-keypair-308857369", "user_data": "}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:48.830779 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] validate_networks() for [('e5ef326a-1def-4c9c-ad55-20a6d9fdbde1', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:57:49.130180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:49.130705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:49.138131 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:49.138624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:49.139082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:49.139780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:49.140236 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:49.140660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:49.166191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lazy-loading 'flavor' on Instance uuid 77aec922-e7ba-4f8c-8931-b2ea825af43c {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:49.207006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:57:49.207430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:57:49.207926 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:57:49.208627 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:57:49.208864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:57:49.219733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:49.220363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:49.220973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:49.221734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:49.222353 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:49.222814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:49.242209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:57:49.246355 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0cdfb590-7f5d-4119-8d3f-20ef9aecb691 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "POST /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c/action" status: 202 len: 0 microversion: 2.1 time: 0.120457 Sep 11 00:57:49.246979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 298/600] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:57:49 2022] POST /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c/action => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 11 00:57:49.248732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Getting quotas for user d886b168a8eb4424987555c073d71727 and project 172e557567404e0f906fd1c3452148d8. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:57:49.262085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:49.262835 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:49.269016 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:49.269731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:49.270336 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:49.271150 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:49.271771 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:49.272203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:49.280222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:57:49.285950 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:57:49.348905 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-688edc22-d89e-4052-8db6-cf9370e6850d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.624585 Sep 11 00:57:49.361777 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 303/601] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:57:48 2022] POST /compute/v2.1/servers => generated 401 bytes in 630 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 11 00:57:49.372464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b5ac0c02-23cb-487b-aa51-9a72671c4548 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:49.373090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b5ac0c02-23cb-487b-aa51-9a72671c4548 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:49.463696 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b5ac0c02-23cb-487b-aa51-9a72671c4548 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:49.471204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b5ac0c02-23cb-487b-aa51-9a72671c4548 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1240 microversion: 2.1 time: 0.107117 Sep 11 00:57:49.475151 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 304/602] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:49 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1240 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:49.482648 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:49.486418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:49.494287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-82561e7a-c070-426d-8476-452d6f66aaad tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.241459 Sep 11 00:57:49.494892 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 299/603] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:49 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:50.495054 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:50.495637 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:50.502663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:50.503185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:50.503620 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:50.504320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:50.504774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:50.505202 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:50.519230 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:50.519925 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:50.529438 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:50.530161 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:50.530814 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:50.531518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:50.531972 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:50.532397 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:50.578212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:50.593802 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-40a8e08d-825c-40de-b74b-790716359111 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1292 microversion: 2.1 time: 0.102645 Sep 11 00:57:50.594536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 305/604] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:50 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1292 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:50.677977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:50.685640 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7313f9dc-085e-4186-96b7-e551a00612b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.170664 Sep 11 00:57:50.686195 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 300/605] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:50 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:51.618084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:51.618715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:51.627562 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:51.628399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:51.629017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:51.629906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:51.630527 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:51.631154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:51.709419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:51.710043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:51.717164 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:51.718052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:51.728585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:51.729373 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:51.729858 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:51.730294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:51.876904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:51.897685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-14d843a3-ec36-4a77-b73d-c447b0fed1cf tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.283159 Sep 11 00:57:51.901107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 306/606] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:51 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:51.932280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:51.942484 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c971335f-0786-4ed6-9fb4-c5c8cb0302a1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.236383 Sep 11 00:57:51.953795 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 301/607] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:51 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:52.412505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-f79f712f-7db9-4721-93cc-ff7c36d35dc3 req-a58b7166-9b94-403e-beb9-026b4060a838 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa6184df-9b60-42df-b8af-547204e85659", "tag": "8ecae1b4-4f80-425a-a9d0-bcd2ab86597f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:52.422524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f79f712f-7db9-4721-93cc-ff7c36d35dc3 req-a58b7166-9b94-403e-beb9-026b4060a838 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:52.423029 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f79f712f-7db9-4721-93cc-ff7c36d35dc3 req-a58b7166-9b94-403e-beb9-026b4060a838 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:52.423461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f79f712f-7db9-4721-93cc-ff7c36d35dc3 req-a58b7166-9b94-403e-beb9-026b4060a838 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:52.427576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:52.440749 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-f79f712f-7db9-4721-93cc-ff7c36d35dc3 req-a58b7166-9b94-403e-beb9-026b4060a838 service nova] Creating event network-changed:8ecae1b4-4f80-425a-a9d0-bcd2ab86597f for instance fa6184df-9b60-42df-b8af-547204e85659 on hv-cinder-maste Sep 11 00:57:52.445844 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-f79f712f-7db9-4721-93cc-ff7c36d35dc3 req-a58b7166-9b94-403e-beb9-026b4060a838 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036900 Sep 11 00:57:52.446652 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 307/608] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:57:52.924291 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:52.924842 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:52.931858 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:52.932343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:52.932777 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:52.933461 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:52.933914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:52.934337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:52.967496 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:52.968292 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:52.974449 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:52.975048 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:52.975544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:52.976484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:52.977179 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:52.977676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:53.124461 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:53.132343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ac0935f4-32c6-4f08-8db2-b875e07d4d49 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.212022 Sep 11 00:57:53.133704 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 302/609] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:52 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:53.152580 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:53.160531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d90cecfa-1495-4199-9e43-f7d2dcb0ebba tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.198176 Sep 11 00:57:53.161090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 308/610] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:52 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:54.155795 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:54.156434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:54.163090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:54.163576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:54.164012 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:54.165354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:54.165809 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:54.166233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:54.183670 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:54.184276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:54.190799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:54.191288 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:54.191715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:54.192406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:54.192853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:54.193272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:54.340230 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:54.347833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4b2055c6-ecc2-42b0-a274-e5e2779890b3 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.195854 Sep 11 00:57:54.348401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 303/611] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:54 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:54.369703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:54.377595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b78e7d73-8395-45a3-8730-e7b57c5ca160 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.197847 Sep 11 00:57:54.378175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 309/612] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:54 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:55.263415 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-674780763"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:55.263953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:55.270206 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.270685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.271148 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.271824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.272274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.272689 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.370636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:55.371404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:55.377940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.378501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.379057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.379818 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.380346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.380852 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.406109 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.network.security_group_api [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Removing security group c7087b87-7128-4e1c-9bb8-3d984694865c from port 497f3d01-c618-4bc4-8866-0f2093427106 Sep 11 00:57:55.608400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:55.616103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d1959554-0010-4341-84b1-1f59febb83b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.249874 Sep 11 00:57:55.616678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 310/613] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:55 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:55.621971 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:55.622575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:55.628337 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.628822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.629253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.629946 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.630400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.630837 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.838829 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:55.846827 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2800913f-208c-45c2-a031-a8e3ee1d3d61 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.228541 Sep 11 00:57:55.847605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 311/614] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:55 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:55.876199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0bf733bf-2126-4702-9026-df7bcc3764c7 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/action" status: 202 len: 0 microversion: 2.1 time: 0.616861 Sep 11 00:57:55.876805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 304/615] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:57:55 2022] POST /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/action => generated 0 bytes in 619 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 11 00:57:55.886485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-9698c4c8-7f55-4bc7-98f3-b692f4afb888 req-9507589c-2321-4224-bb4e-f4d7d28ffc2d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:55.896868 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-9698c4c8-7f55-4bc7-98f3-b692f4afb888 req-9507589c-2321-4224-bb4e-f4d7d28ffc2d service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:55.897152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-9698c4c8-7f55-4bc7-98f3-b692f4afb888 req-9507589c-2321-4224-bb4e-f4d7d28ffc2d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:55.897565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-9698c4c8-7f55-4bc7-98f3-b692f4afb888 req-9507589c-2321-4224-bb4e-f4d7d28ffc2d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:55.909692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-9698c4c8-7f55-4bc7-98f3-b692f4afb888 req-9507589c-2321-4224-bb4e-f4d7d28ffc2d service nova] Creating event network-changed:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:57:55.915461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-9698c4c8-7f55-4bc7-98f3-b692f4afb888 req-9507589c-2321-4224-bb4e-f4d7d28ffc2d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032986 Sep 11 00:57:55.916357 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 312/616] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:56.640747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:56.641484 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:56.647431 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:56.647914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:56.649144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:56.649870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:56.650344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:56.650753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:56.849827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:56.858440 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ea6a8109-f4e3-434c-9889-b35e50d5f4f6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.222047 Sep 11 00:57:56.859993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 305/617] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:56 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:56.871626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:56.872446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:56.875930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=76,nova_api:INSERT=6,nova_api:DELETE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:56.880541 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:56.881021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:56.881449 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:56.882163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:56.882658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:56.884122 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:56.982193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:56.983009 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:56.991489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:56.991984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:56.992958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:56.993662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:56.994123 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:56.994547 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:57.048713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:57.057655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-58841b6f-8efe-4c47-93f0-7c577c735b5e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.190973 Sep 11 00:57:57.058224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 313/618] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:56 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:57:57.082923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lazy-loading 'services' on Instance uuid 95a9cb8e-523c-49b2-ae9e-de849b956dc7 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:57:57.157390 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-78d50d51-2311-470f-9a56-9e5f05c9a048 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments/5b00b1ee-8b83-4674-bf79-ae77aef74879" status: 202 len: 0 microversion: 2.1 time: 0.178993 Sep 11 00:57:57.158341 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 306/619] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:57:56 2022] DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments/5b00b1ee-8b83-4674-bf79-ae77aef74879 => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:57:57.167080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:57.169227 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:57.175193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:57.175677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:57.176103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:57.176790 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:57.177233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:57.177649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:57.208316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6a58b5a5-3386-4689-937f-dfab87764f80 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045644 Sep 11 00:57:57.209068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 314/620] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:57 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:57.881181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=4 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:57.886475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:57.887249 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:57.893821 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:57.894547 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:57.895139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:57.895834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:57.896287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:57.896709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:57.938495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-3078185a-8eeb-4ffe-a6cb-0fec874f9c0a req-20551f7f-6080-4c00-9cc5-ade2ccf4018b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:57:57.950493 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3078185a-8eeb-4ffe-a6cb-0fec874f9c0a req-20551f7f-6080-4c00-9cc5-ade2ccf4018b service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:57.951146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3078185a-8eeb-4ffe-a6cb-0fec874f9c0a req-20551f7f-6080-4c00-9cc5-ade2ccf4018b service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:57.951686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-3078185a-8eeb-4ffe-a6cb-0fec874f9c0a req-20551f7f-6080-4c00-9cc5-ade2ccf4018b service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:57.966553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-3078185a-8eeb-4ffe-a6cb-0fec874f9c0a req-20551f7f-6080-4c00-9cc5-ade2ccf4018b service nova] Creating event network-changed:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:57:57.971937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-3078185a-8eeb-4ffe-a6cb-0fec874f9c0a req-20551f7f-6080-4c00-9cc5-ade2ccf4018b service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038172 Sep 11 00:57:57.973200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 315/621] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:57:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:58.087706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:58.088486 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:58.094031 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:58.095215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:58.095712 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:58.097593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:58.097593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:58.097593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:58.097908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:58.114560 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-712c0827-8697-4fcf-bf89-69f3556aed1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.234071 Sep 11 00:57:58.115958 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 307/622] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:57 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:58.222494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:58.227776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:58.236953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:58.237895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:58.238900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:58.239859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:58.241272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:58.241963 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:58.304030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-58d937b5-c902-49f9-895d-e607337bc617 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086127 Sep 11 00:57:58.315313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 308/623] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:58 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:57:58.319782 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:58.328541 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-405f6289-ad8e-424a-ae94-21d49f828594 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.246134 Sep 11 00:57:58.335728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 316/624] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:58 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:59.146002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:59.147439 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:59.153882 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.154378 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.155168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.155954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.156453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.156918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.283428 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:59.291770 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c3bfd810-8aa5-4d2c-8cc0-decbe3f1e6f2 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.150210 Sep 11 00:57:59.292325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 309/625] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:59 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:57:59.322450 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:59.324655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:59.327120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:59.332764 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.333251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.333719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.334565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.335289 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.336258 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.353669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:59.354374 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:59.360636 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.361198 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.361875 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.362733 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.363287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.363712 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.370062 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5d75b274-c526-4cc7-a60f-882e3478f767 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051663 Sep 11 00:57:59.370632 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 317/626] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:57:59 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 00:57:59.381863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:57:59.382362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:57:59.388753 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.389240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.389677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.390371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:57:59.390843 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:57:59.391270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:57:59.467968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=8 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:59.469138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] HTTP exception thrown: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 is not attached to volume 5b00b1ee-8b83-4674-bf79-ae77aef74879 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Request method failure captured: Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments/5b00b1ee-8b83-4674-bf79-ae77aef74879 HTTP/1.1 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTHexHRRwNGeQ-QzdLw8w1gN8ky1cfX9ngWduS9_DeSoeLjyyigbKWdSdnR421q_9XLCFu4ZtjghiitVYMFHVi-TAekvFepS70nHlBifNU98eOpW4KFsZS0Zs7irUV041J_G6SyElm22-qRyfH7ncQfEh8J7Q_DsrJuEX9DKZMDX3B8 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 299fcf2ee8b24a4fbb27ed83954c8e24 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-TestMinimumBasicScenario-490773617 Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:57:59.471610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-TestMinimumBasicScenario-490773617 Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 299fcf2ee8b24a4fbb27ed83954c8e24 Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-TestMinimumBasicScenario-490773617 Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-TestMinimumBasicScenario-490773617-project Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 953d7ed2081c48c79a1a5660370b861b Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-TestMinimumBasicScenario-490773617-project Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:57:59.473165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 is not attached to volume 5b00b1ee-8b83-4674-bf79-ae77aef74879 Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'server_id': '95a9cb8e-523c-49b2-ae9e-de849b956dc7', 'id': '5b00b1ee-8b83-4674-bf79-ae77aef74879'} Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 5b00b1ee-8b83-4674-bf79-ae77aef74879. Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:57:59.473912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 is not attached to volume 5b00b1ee-8b83-4674-bf79-ae77aef74879 Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 is not attached to volume 5b00b1ee-8b83-4674-bf79-ae77aef74879 Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:57:59.475245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Returning 404 to user: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 is not attached to volume 5b00b1ee-8b83-4674-bf79-ae77aef74879 {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:57:59.476357 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f936dd21-a9b0-4c84-8727-1bf8512dc40b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments/5b00b1ee-8b83-4674-bf79-ae77aef74879" status: 404 len: 154 microversion: 2.1 time: 0.095767 Sep 11 00:57:59.476357 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 318/627] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:57:59 2022] DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7/os-volume_attachments/5b00b1ee-8b83-4674-bf79-ae77aef74879 => generated 154 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 00:57:59.503718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:57:59.512396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0622a395-f6b6-4572-a275-8a8398e83c75 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.163067 Sep 11 00:57:59.513045 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 310/628] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:57:59 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:00.314715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:00.315492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:00.322164 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.322707 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.323326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.323854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.324356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.324776 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.459251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:00.467375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a277f568-54de-41c1-ad4c-28681e1b4151 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.156509 Sep 11 00:58:00.467890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 319/629] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:00 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:00.538378 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:00.539666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:00.547579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.548024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.549729 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.550497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.551150 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.551581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.629804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:00.630325 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:00.636601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.637089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.637726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.638604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.639084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.639571 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.667149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:58:00.716021 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:00.724453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c64f1823-7a4b-4216-888e-d4dfe55644af tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.190473 Sep 11 00:58:00.739421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 311/630] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:00 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:00.755521 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ad65b105-93e5-4e83-9e9a-7a38e4849791 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 204 len: 0 microversion: 2.1 time: 0.133933 Sep 11 00:58:00.756089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 320/631] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:58:00 2022] DELETE /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 00:58:00.765283 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:00.765901 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:00.771945 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.772435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.772870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.774212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:00.774673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:00.775128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:00.907243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:00.915622 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-68f228a2-eb9d-4c9b-b348-bd8d6b2478cb tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1553 microversion: 2.1 time: 0.154299 Sep 11 00:58:00.916178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 312/632] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:00 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1553 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:01.546481 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:01.547202 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:01.553102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:01.553573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:01.553995 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:01.555027 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:01.555534 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:01.555956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:01.684406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:01.692080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-184d1b7b-ce93-4f3d-9bc6-936e459a5e81 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.204633 Sep 11 00:58:01.692647 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 321/633] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:01 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:01.751737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:01.752276 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:01.754641 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:01.761169 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:01.761650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:01.762085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:01.762765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:01.763237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:01.763658 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:01.897126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:01.906593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f7388d95-7771-4c56-ad65-7c9b2191e788 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.158923 Sep 11 00:58:01.907212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 313/634] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:01 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:01.939345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:01.940020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:01.946338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:01.946827 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:01.947252 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:01.947930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:01.948375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:01.948790 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:02.078200 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:02.087335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5f9991c1-361b-4c3f-8294-9b687ac5a462 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1553 microversion: 2.1 time: 0.151719 Sep 11 00:58:02.087908 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 322/635] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:01 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1553 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:02.719122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:02.719122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:02.722954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:02.723514 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:02.724099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:02.724952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:02.725473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:02.725963 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:02.930087 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:02.930867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:02.937904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:02.939754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:02.940927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:02.941427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:02.941856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:02.942528 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:02.943214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:02.951209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3384fc6e-50ff-4ea1-b2f3-76bdba433120 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.239282 Sep 11 00:58:02.952416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 314/636] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:02 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:03.113193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:03.113814 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:03.126166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:03.126656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:03.127313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:03.128327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:03.129020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:03.129522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:03.158763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:03.168139 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1e95e1bf-39be-4940-ad0d-bca5400308e8 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.242510 Sep 11 00:58:03.168714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 323/637] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:02 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:03.330924 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:03.339330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-048c17d5-6780-4823-be01-ff01d26a121c tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1553 microversion: 2.1 time: 0.229879 Sep 11 00:58:03.339905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 315/638] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:03 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1553 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:03.457225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-d1636af4-071d-411e-953f-2f42138c3fea req-a61fd409-ec8e-4721-9bf5-ea8448aafae3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95a9cb8e-523c-49b2-ae9e-de849b956dc7", "name": "network-vif-deleted", "tag": "497f3d01-c618-4bc4-8866-0f2093427106"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:03.467319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d1636af4-071d-411e-953f-2f42138c3fea req-a61fd409-ec8e-4721-9bf5-ea8448aafae3 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:03.467792 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d1636af4-071d-411e-953f-2f42138c3fea req-a61fd409-ec8e-4721-9bf5-ea8448aafae3 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:03.468222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d1636af4-071d-411e-953f-2f42138c3fea req-a61fd409-ec8e-4721-9bf5-ea8448aafae3 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:03.471940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:03.479636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-d1636af4-071d-411e-953f-2f42138c3fea req-a61fd409-ec8e-4721-9bf5-ea8448aafae3 service nova] Creating event network-vif-deleted:497f3d01-c618-4bc4-8866-0f2093427106 for instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 on hv-cinder-maste Sep 11 00:58:03.484624 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-d1636af4-071d-411e-953f-2f42138c3fea req-a61fd409-ec8e-4721-9bf5-ea8448aafae3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031024 Sep 11 00:58:03.485248 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 324/639] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:03.981092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:03.982244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:03.989845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:03.990526 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:03.991172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:03.992292 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:03.992948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:03.993569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:03.997965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:04.079323 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-35cc705c-aa90-4019-a193-04a50885d97a req-8c88ade5-136f-4572-831b-8ac61a4fd2e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa6184df-9b60-42df-b8af-547204e85659", "name": "network-vif-plugged", "status": "completed", "tag": "8ecae1b4-4f80-425a-a9d0-bcd2ab86597f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:04.091917 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-35cc705c-aa90-4019-a193-04a50885d97a req-8c88ade5-136f-4572-831b-8ac61a4fd2e7 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:04.092986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-35cc705c-aa90-4019-a193-04a50885d97a req-8c88ade5-136f-4572-831b-8ac61a4fd2e7 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:04.093998 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-35cc705c-aa90-4019-a193-04a50885d97a req-8c88ade5-136f-4572-831b-8ac61a4fd2e7 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:04.107257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-35cc705c-aa90-4019-a193-04a50885d97a req-8c88ade5-136f-4572-831b-8ac61a4fd2e7 service nova] Creating event network-vif-plugged:8ecae1b4-4f80-425a-a9d0-bcd2ab86597f for instance fa6184df-9b60-42df-b8af-547204e85659 on hv-cinder-maste Sep 11 00:58:04.112254 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-35cc705c-aa90-4019-a193-04a50885d97a req-8c88ade5-136f-4572-831b-8ac61a4fd2e7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036894 Sep 11 00:58:04.113493 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 325/640] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:58:04.148811 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:04.157042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d8f44709-d7f3-4465-912c-079e21496740 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.185263 Sep 11 00:58:04.157881 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 316/641] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:03 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:04.192256 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:04.192947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:04.200377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:04.200864 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:04.201296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:04.201987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:04.202439 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:04.202945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:04.333065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:04.341410 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f50178b7-2864-458b-abf7-7ebcb044e076 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.153357 Sep 11 00:58:04.341974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 326/642] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:04 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:04.362402 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:04.363044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:04.369176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:04.369658 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:04.370095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:04.370780 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:04.371259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:04.371682 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:04.431952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:04.439548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e35d6e4d-8bb3-4878-81b2-dd94b6544e2a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1511 microversion: 2.1 time: 0.081070 Sep 11 00:58:04.440111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 317/643] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:04 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1511 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:05.180946 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:05.181557 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:05.187863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:05.188339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:05.188792 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:05.189462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:05.189904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:05.190321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:05.365166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:05.365729 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:05.373191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:05.373688 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:05.374120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:05.374841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:05.375294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:05.375718 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:05.598230 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:05.606405 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4071e578-79a6-4602-a343-a9d5205043b5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.430037 Sep 11 00:58:05.607005 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 327/644] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:05 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:05.607762 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:05.612388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:05.613164 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:05.615663 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1a437816-251a-46d1-be51-c287c58bff52 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.254456 Sep 11 00:58:05.616165 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 318/645] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:05 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:05.622250 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:05.622732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:05.623198 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:05.623891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:05.624343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:05.624762 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:05.687171 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:05.694718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ec5343e7-4a44-4003-be6d-a47ec6648f7b tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1511 microversion: 2.1 time: 0.085935 Sep 11 00:58:05.695666 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 328/646] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:05 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1511 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:06.629769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:06.630388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:06.637647 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:06.638129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:06.638572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:06.639286 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:06.639399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:06.639789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:06.640019 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:06.640159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:06.646448 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:06.646959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:06.647390 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:06.648135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:06.648646 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:06.649067 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:06.842304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:06.846623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:06.850928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-465133dc-ea33-4732-ad01-8e9c18fd52ec tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.224941 Sep 11 00:58:06.851699 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 319/647] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:06 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:06.856725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:06.857452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:06.863179 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:06.864140 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:06.864140 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:06.864903 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:06.865400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:06.865942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:06.871670 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cc79a1ea-6449-4d6e-9680-0ae0d814b55e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.235963 Sep 11 00:58:06.872830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 329/648] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:06 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:06.930056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:06.937632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-317074c7-19ab-4b07-8a42-c4c89ff135ae tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1342 microversion: 2.1 time: 0.085191 Sep 11 00:58:06.938192 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 320/649] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:06 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1342 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:07.879836 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:07.880458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:07.887450 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:07.887937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:07.888369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:07.889061 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:07.889513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:07.889937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:07.896642 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:07.897396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:07.904822 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:07.905311 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:07.905747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:07.906442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:07.906948 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:07.907390 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:08.087898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:08.092166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:08.095567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-469b03a5-1674-46a9-b9e0-d2ce2ab9a620 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1365 microversion: 2.1 time: 0.219615 Sep 11 00:58:08.096146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 330/650] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:07 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1365 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:08.100248 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5544ec59-b85c-416c-be99-1ff8886c85e5 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.207365 Sep 11 00:58:08.107523 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 321/651] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:07 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:08.108222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:08.108222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:08.110366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:08.110891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:08.111329 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:08.112024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:08.112480 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:08.112939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:08.145906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:08.146395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:08.146855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:08.147304 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lazy-loading 'fault' on Instance uuid 95a9cb8e-523c-49b2-ae9e-de849b956dc7 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:58:08.188697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '953d7ed2081c48c79a1a5660370b861b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '299fcf2ee8b24a4fbb27ed83954c8e24', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:08.197102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8dac072e-3f8d-4805-a3c9-2f0baefba3aa tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 200 len: 1347 microversion: 2.1 time: 0.099962 Sep 11 00:58:08.197675 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 331/652] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:08 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 1347 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:09.118984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:09.119600 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:09.123786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:09.124431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:09.126670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.127194 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.127630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.128326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.128806 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.129231 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.130751 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.131251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.131676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.132498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.133216 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.133645 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.326271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:09.330102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:09.334898 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-af0dae2e-ce26-4633-9384-a918631be663 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1555 microversion: 2.1 time: 0.219643 Sep 11 00:58:09.336418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 322/653] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:09 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1555 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:09.338816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6d3870e0-c3ab-499d-8eb4-54141ad0e47b tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1570 microversion: 2.1 time: 0.219434 Sep 11 00:58:09.339369 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 332/654] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:09 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1570 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:09.341709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:09.342424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] [instance: 95a9cb8e-523c-49b2-ae9e-de849b956dc7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:09.348500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.349362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.350027 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.351297 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.351996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.352638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.360198 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:09.360941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:09.368006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] HTTP exception thrown: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 could not be found. Sep 11 00:58:09.368308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.368892 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.369449 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.370148 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Request method failure captured: Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 HTTP/1.1 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTHexHRRwNGeQ-QzdLw8w1gN8ky1cfX9ngWduS9_DeSoeLjyyigbKWdSdnR421q_9XLCFu4ZtjghiitVYMFHVi-TAekvFepS70nHlBifNU98eOpW4KFsZS0Zs7irUV041J_G6SyElm22-qRyfH7ncQfEh8J7Q_DsrJuEX9DKZMDX3B8 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 299fcf2ee8b24a4fbb27ed83954c8e24 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-TestMinimumBasicScenario-490773617 Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:58:09.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_299fcf2ee8b24a4fbb27ed83954c8e24"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-TestMinimumBasicScenario-490773617 Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 299fcf2ee8b24a4fbb27ed83954c8e24 Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-TestMinimumBasicScenario-490773617 Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-TestMinimumBasicScenario-490773617-project Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 953d7ed2081c48c79a1a5660370b861b Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-TestMinimumBasicScenario-490773617-project Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 could not be found. Sep 11 00:58:09.372173 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '95a9cb8e-523c-49b2-ae9e-de849b956dc7'} Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 could not be found. Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:58:09.373121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 could not be found. Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 could not be found. Sep 11 00:58:09.376410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:58:09.377915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.377915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.378097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Returning 404 to user: Instance 95a9cb8e-523c-49b2-ae9e-de849b956dc7 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:58:09.378097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-77d2b3c2-7ca7-4efa-b128-15d9e8782610 tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7" status: 404 len: 111 microversion: 2.1 time: 0.037283 Sep 11 00:58:09.378097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 323/655] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:09 2022] GET /compute/v2.1/servers/95a9cb8e-523c-49b2-ae9e-de849b956dc7 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:58:09.383156 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:09.383836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:09.390325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.390927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.391454 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.392218 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.392747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.393247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.424440 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=126979) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 11 00:58:09.456737 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-af494e15-4ad9-40bc-8b84-0b874cbb9ca6 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.100418 Sep 11 00:58:09.457495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 333/656] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 00:58:09 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-interface => generated 284 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:58:09.463803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-77e915a0-4c04-4b30-b886-8979d530319a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] Calling method '.version_select at 0x7fc01ac588b0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:09.486264 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-77e915a0-4c04-4b30-b886-8979d530319a tempest-TestMinimumBasicScenario-490773617 tempest-TestMinimumBasicScenario-490773617-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-371702567" status: 202 len: 0 microversion: 2.1 time: 0.027449 Sep 11 00:58:09.487196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 334/657] 10.222.0.27 () {58 vars in 1269 bytes} [Sun Sep 11 00:58:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-371702567 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:58:09.495453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c842af49-5d25-4a64-a3e4-8430d0bc6b13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "POST /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c/action" status: 202 len: 0 microversion: 2.1 time: 0.118325 Sep 11 00:58:09.496348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 324/658] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 00:58:09 2022] POST /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:58:09.504531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:09.505371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:09.512401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.512885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.513765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.514055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:09.514441 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:09.514846 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:09.761101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:09.769206 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e5426e8b-ec5f-416c-8680-c4ef4af73de7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1518 microversion: 2.1 time: 0.268717 Sep 11 00:58:09.770340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 335/659] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:09 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1518 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:10.165747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-b1f60713-78de-4361-9cb7-92b8513fde32 req-5931b7af-7003-4490-a43a-3122e6513870 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "name": "network-vif-unplugged", "status": "completed", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:10.182710 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-b1f60713-78de-4361-9cb7-92b8513fde32 req-5931b7af-7003-4490-a43a-3122e6513870 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:10.183894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-b1f60713-78de-4361-9cb7-92b8513fde32 req-5931b7af-7003-4490-a43a-3122e6513870 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:10.184953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-b1f60713-78de-4361-9cb7-92b8513fde32 req-5931b7af-7003-4490-a43a-3122e6513870 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:10.212753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.compute.server_external_events [req-b1f60713-78de-4361-9cb7-92b8513fde32 req-5931b7af-7003-4490-a43a-3122e6513870 service nova] Unable to find a host for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c. Dropping event network-vif-unplugged {{(pid=126979) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 11 00:58:10.213974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-b1f60713-78de-4361-9cb7-92b8513fde32 req-5931b7af-7003-4490-a43a-3122e6513870 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.057379 Sep 11 00:58:10.214965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 325/660] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:10 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 60 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Sep 11 00:58:10.792784 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:10.793398 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:10.799871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:10.800348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:10.800985 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:10.801841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:10.802294 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:10.802717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:11.168102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:11.181587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-511286aa-2f7a-4deb-a316-3725bb20a5fa tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1520 microversion: 2.1 time: 0.392383 Sep 11 00:58:11.182545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 336/661] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:10 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1520 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:11.505435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-a2364464-4d4a-4e96-b04c-695f5eb1a31c req-99e37a6e-f9d7-48b8-9dd7-c1dd7e73b331 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa6184df-9b60-42df-b8af-547204e85659", "tag": "8ecae1b4-4f80-425a-a9d0-bcd2ab86597f"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:11.527304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a2364464-4d4a-4e96-b04c-695f5eb1a31c req-99e37a6e-f9d7-48b8-9dd7-c1dd7e73b331 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:11.527790 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a2364464-4d4a-4e96-b04c-695f5eb1a31c req-99e37a6e-f9d7-48b8-9dd7-c1dd7e73b331 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:11.528713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-a2364464-4d4a-4e96-b04c-695f5eb1a31c req-99e37a6e-f9d7-48b8-9dd7-c1dd7e73b331 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:11.574296 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-a2364464-4d4a-4e96-b04c-695f5eb1a31c req-99e37a6e-f9d7-48b8-9dd7-c1dd7e73b331 service nova] Creating event network-changed:8ecae1b4-4f80-425a-a9d0-bcd2ab86597f for instance fa6184df-9b60-42df-b8af-547204e85659 on hv-cinder-maste Sep 11 00:58:11.592683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-a2364464-4d4a-4e96-b04c-695f5eb1a31c req-99e37a6e-f9d7-48b8-9dd7-c1dd7e73b331 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.429677 Sep 11 00:58:11.593878 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 326/662] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:12.206712 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:12.207472 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:12.223222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:12.223730 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:12.224171 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:12.224885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:12.225341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:12.227757 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:12.545492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:12.559158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f76a4953-eae3-46cf-9806-1eac4b8e2d13 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.357173 Sep 11 00:58:12.559748 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 337/663] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:12 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:13.586769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:13.588478 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:13.596716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:13.597473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:13.598168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:13.599141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:13.599922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:13.600552 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:13.774849 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:13.785600 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9e2023f8-1a87-422a-851c-0e8e6cb7acf1 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.204068 Sep 11 00:58:13.786711 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 327/664] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:13 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:14.811915 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:14.812536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:14.824851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:14.825347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:14.827053 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:14.828059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:14.828737 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:14.829715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:15.101371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:15.123103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ec3b69bd-0cac-42d3-88ca-a23bf80c3e12 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.316613 Sep 11 00:58:15.135652 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 338/665] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:14 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:16.149472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:16.150650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:16.160029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:16.160778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:16.161445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:16.162335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:16.162982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:16.163683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:16.342921 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:16.352206 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-938907a1-3da0-4243-9ab9-35a0a3603c6d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.207600 Sep 11 00:58:16.352786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 328/666] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:16 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:17.378910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:17.380193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:17.388033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:17.388546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:17.388989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:17.389703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:17.390160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:17.390585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:17.870656 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:17.881060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f7bf5537-4822-4b34-a869-92c91a14d8fc tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.508520 Sep 11 00:58:17.882495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 339/667] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:17 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:18.905144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:18.905758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:18.912416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:18.912935 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:18.914070 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:18.914828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:18.915287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:18.915942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:19.070886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:19.073951 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-328cf7db-48e8-4256-b3bb-535ed3d47704 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.170695 Sep 11 00:58:19.073951 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 329/668] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:18 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:20.094506 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:20.095148 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:20.101815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:20.102303 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:20.102734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:20.103538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:20.103994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:20.104416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:20.245372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:20.259553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-adaa7c55-fde5-464d-926c-c86da1db430e tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.168774 Sep 11 00:58:20.260141 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 340/669] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:20 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:21.283558 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:21.284166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:21.291586 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:21.292061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:21.292488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:21.293166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:21.293614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:21.294030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:21.425473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:21.433840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-89a29e43-80bf-4568-bc23-1ee9442a2b70 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.154428 Sep 11 00:58:21.434399 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 330/670] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:21 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:22.340343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-204858ee-badf-4f51-9936-9ce7bbf5da1c req-e88518f5-014e-4263-8282-d5fdaa2220f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "name": "network-vif-plugged", "status": "completed", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:22.350130 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-204858ee-badf-4f51-9936-9ce7bbf5da1c req-e88518f5-014e-4263-8282-d5fdaa2220f4 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:22.351172 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-204858ee-badf-4f51-9936-9ce7bbf5da1c req-e88518f5-014e-4263-8282-d5fdaa2220f4 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:22.351853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-204858ee-badf-4f51-9936-9ce7bbf5da1c req-e88518f5-014e-4263-8282-d5fdaa2220f4 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:22.364775 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-204858ee-badf-4f51-9936-9ce7bbf5da1c req-e88518f5-014e-4263-8282-d5fdaa2220f4 service nova] Creating event network-vif-plugged:549957cc-ef50-4e12-9608-80b215c4bfb4 for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c on hv-cinder-maste Sep 11 00:58:22.370683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-204858ee-badf-4f51-9936-9ce7bbf5da1c req-e88518f5-014e-4263-8282-d5fdaa2220f4 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.185790 Sep 11 00:58:22.371450 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 341/671] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:22.461085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:22.461085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:22.465446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:22.466202 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:22.467035 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:22.468005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:22.468563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:22.469227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:22.615504 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:22.624350 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-98f639ee-1ccb-4781-9d13-78ff6229da3f tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.171929 Sep 11 00:58:22.624933 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 331/672] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:22 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:23.650483 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:23.651306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:23.657941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:23.658427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:23.658878 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:23.659567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:23.660018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:23.660441 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:23.827260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:23.836060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3d626e68-5199-4cc1-97dc-0bc51c7e8ba9 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1576 microversion: 2.1 time: 0.191133 Sep 11 00:58:23.836770 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 342/673] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:23 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1576 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:24.859645 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:24.860618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:24.868150 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:24.868797 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:24.869332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:24.870175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:24.870928 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:24.871461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:25.026380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:25.027110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:25.027239 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:25.030860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:25.034223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:25.034869 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:25.035387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:25.036143 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:25.036259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cc6844d8-9885-4c58-8675-2ac7f530d7b7 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1563 microversion: 2.1 time: 0.180636 Sep 11 00:58:25.036707 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 343/674] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:24 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1563 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:25.036843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:25.037235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:25.061091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d5ea5090-5adc-434d-9e3e-2186583c5c38 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/ips" status: 200 len: 151 microversion: 2.20 time: 0.275789 Sep 11 00:58:25.061591 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 332/675] 10.222.0.27 () {60 vars in 1289 bytes} [Sun Sep 11 00:58:24 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/ips => generated 151 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:58:26.865332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:26.866046 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:26.872801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:26.873282 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:26.873715 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:26.874453 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:26.874939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:26.875362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:26.897710 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'flavor' on Instance uuid 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:58:27.083889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3b966c29-2539-4de5-bdae-1f63ba210e03 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/action" status: 202 len: 0 microversion: 2.20 time: 0.222205 Sep 11 00:58:27.085037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 344/676] 10.222.0.27 () {62 vars in 1319 bytes} [Sun Sep 11 00:58:26 2022] POST /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/action => generated 0 bytes in 224 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:58:27.093859 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:27.094476 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:27.101861 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:27.102275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:27.102685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:27.103422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:27.103935 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:27.104376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:27.239655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:27.249014 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:27.250495 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:27.251699 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-86f8f8e9-d97f-45e1-bc5d-2ec3dc118574 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.162048 Sep 11 00:58:27.252257 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 333/677] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:27 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:28.279327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:28.280867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:28.287253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:28.295699 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:28.296481 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:28.297274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:28.298345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:28.299004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:28.299576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:28.442596 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:28.452065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:28.453462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:28.454631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5c94d4b1-1c35-464d-baa8-c3b4649c60b2 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.180569 Sep 11 00:58:28.455231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 345/678] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:28 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:29.480166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:29.480786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:29.487531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:29.487991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:29.488392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:29.489344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:29.489807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:29.490235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:29.494548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:29.687767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:29.696613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:29.698066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:29.699254 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4ca1032f-287f-490b-be92-6bf67e04ea7a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.222996 Sep 11 00:58:29.699933 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 334/679] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:29 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:30.724271 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:30.725208 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:30.731917 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:30.732405 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:30.732846 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:30.733547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:30.734001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:30.734425 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:30.870742 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:30.878631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:30.880073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:30.881580 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-149343f5-243f-4aba-b168-dc8d53ddf483 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.161026 Sep 11 00:58:30.882584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 346/680] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:30 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:31.908305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:31.908906 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:31.911177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:31.917020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:31.917503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:31.917935 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:31.918620 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:31.919092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:31.919517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:32.051334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:32.059997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:32.061426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:32.062583 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6a0273f5-f2ca-4c19-afae-472e268c0898 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.158393 Sep 11 00:58:32.063190 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 335/681] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:31 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:33.087622 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:33.088367 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:33.094763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:33.095291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:33.095731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:33.096426 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:33.096881 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:33.097308 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:33.224181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:33.232869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:33.234301 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:33.235735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-05007ae5-f81b-4b3f-a085-0c806c4e484f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.152446 Sep 11 00:58:33.236294 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 347/682] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:33 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:58:34.263272 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:34.265182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:34.272231 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:34.272668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:34.273379 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:34.274073 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:34.274522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:34.274968 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:34.410816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:34.419118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:34.420679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:34.421928 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-91fc32dc-7406-4556-bcdd-74451c0c4971 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.163083 Sep 11 00:58:34.422564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 336/683] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:34 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:35.448372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:35.449113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:35.455785 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:35.456372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:35.456840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:35.457515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:35.457975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:35.458407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:35.593510 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:35.601613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:35.603158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:35.604319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9154442f-521a-4b09-9a4b-31b71d6010e7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.160577 Sep 11 00:58:35.604880 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 348/684] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:35 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:36.629512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:36.630224 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:36.636602 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:36.637168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:36.637613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:36.638308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:36.638772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:36.639222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:36.776095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:36.784696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:36.786216 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:36.787509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1b7b53c5-a32e-4e94-aa64-9edec696e7ca tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.161962 Sep 11 00:58:36.788874 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 337/685] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:36 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:37.814938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:37.815579 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:37.822347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:37.822841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:37.823274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:37.823948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:37.824393 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:37.824805 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:37.952564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:37.961278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:37.962713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:37.963898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2c62a8e0-d2cf-41af-afbe-2170d09408ee tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.153000 Sep 11 00:58:37.964446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 349/686] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:37 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:38.505133 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:38.505992 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:38.513581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:38.514121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:38.514634 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:38.515377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:38.515879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:38.516477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:38.545529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:58:38.630139 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d40e0ff4-e21c-43fa-ab93-538d1143af32 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "DELETE /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 204 len: 0 microversion: 2.1 time: 0.130379 Sep 11 00:58:38.630891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 338/687] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:58:38 2022] DELETE /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 00:58:38.640702 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:38.641352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:38.648543 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:38.649030 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:38.649462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:38.650542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:38.651033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:38.651511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:38.786576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:38.795137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a3ede09f-a7d2-41c5-bb88-21fcf5fddde8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1575 microversion: 2.1 time: 0.159069 Sep 11 00:58:38.795697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 350/688] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:38 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1575 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:38.990190 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:38.991446 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:38.997398 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:38.997876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:38.998334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:38.999042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:38.999493 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:38.999917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:39.132509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:39.141611 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:39.143290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:39.144623 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4ca7429d-0ee3-4a3f-b0fe-a1c874c12963 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.158712 Sep 11 00:58:39.145216 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 339/689] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:38 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:39.818555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:39.819192 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:39.825770 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:39.826254 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:39.826689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:39.827403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:39.827860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:39.828293 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:39.962961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:39.970926 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b56c6f4e-d74e-46b6-8f32-1b3c81122415 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1575 microversion: 2.1 time: 0.156185 Sep 11 00:58:39.971482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 351/690] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:39 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1575 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:40.169883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:40.170316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:40.177024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:40.177562 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:40.177792 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:40.178490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:40.179048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:40.179709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:40.328415 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:40.336802 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:40.338243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:40.339503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d346710d-5f34-4ff7-841f-829f60f3ec43 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.173630 Sep 11 00:58:40.340071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 340/691] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:40 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:40.994160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:40.994807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:41.001177 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:41.001894 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:41.002335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:41.003059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:41.003514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:41.003942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:41.183026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:41.190645 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-735188ab-0aeb-40fe-8da0-0b90384d2703 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1575 microversion: 2.1 time: 0.201121 Sep 11 00:58:41.191241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 352/692] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:40 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1575 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:41.365367 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:41.366697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:41.373703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:41.374187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:41.374614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:41.375327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:41.375775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:41.376215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:41.587121 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-d5e8ca6e-9c51-482e-8f01-2f8140ca2047 req-e3a59cfa-020b-43ed-8e87-561aa27da20a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "984863bd-b3a1-4546-8b36-720ba324d7ac", "name": "network-vif-deleted", "tag": "883cbb85-cb19-4ae7-86be-1e7abfa4f1f3"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:41.606075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d5e8ca6e-9c51-482e-8f01-2f8140ca2047 req-e3a59cfa-020b-43ed-8e87-561aa27da20a service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:41.606974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d5e8ca6e-9c51-482e-8f01-2f8140ca2047 req-e3a59cfa-020b-43ed-8e87-561aa27da20a service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:41.614777 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d5e8ca6e-9c51-482e-8f01-2f8140ca2047 req-e3a59cfa-020b-43ed-8e87-561aa27da20a service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:41.631705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:41.634487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-d5e8ca6e-9c51-482e-8f01-2f8140ca2047 req-e3a59cfa-020b-43ed-8e87-561aa27da20a service nova] Creating event network-vif-deleted:883cbb85-cb19-4ae7-86be-1e7abfa4f1f3 for instance 984863bd-b3a1-4546-8b36-720ba324d7ac on hv-cinder-maste Sep 11 00:58:41.644315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-d5e8ca6e-9c51-482e-8f01-2f8140ca2047 req-e3a59cfa-020b-43ed-8e87-561aa27da20a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063406 Sep 11 00:58:41.645532 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 353/693] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:41.650573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:41.653041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:41.654418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-51fc13c5-2edb-413b-abf5-3e49c7c6d28b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.292935 Sep 11 00:58:41.655034 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 341/694] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:41 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:42.222587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:42.223575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:42.231286 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:42.231799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:42.232224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:42.233470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:42.234112 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:42.234748 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:42.268116 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:42.268829 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:42.269419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:42.270021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lazy-loading 'fault' on Instance uuid 984863bd-b3a1-4546-8b36-720ba324d7ac {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:58:42.312280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:42.319839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c0ac57e2-01d4-411c-a9db-2091ed233d49 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 200 len: 1229 microversion: 2.1 time: 0.107656 Sep 11 00:58:42.321306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 354/695] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:42 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 1229 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:42.683035 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:42.683634 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:42.689919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:42.690405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:42.690870 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:42.691561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:42.692043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:42.692469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:43.008716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:43.019265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:43.021585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:43.023955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a6017d2d-c1c8-4030-bac3-26ba58f8ec02 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1745 microversion: 2.20 time: 0.344690 Sep 11 00:58:43.024887 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 342/696] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:42 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1745 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:43.344591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:43.345345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:43.351960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:43.352443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:43.352869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:43.353555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:43.354009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:43.354423 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:43.368103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] HTTP exception thrown: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Request method failure captured: Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac HTTP/1.1 Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTHw8BzyvxS9fdqb226qp72rJVPqsBknMqx-aQjJYwTnPCIl2mNNo1zdJdKfwd7rEQQXjZPG3rPu9YXC3e4Bo9Vda4P2ZVOndGogkzl1WxCa_0jqx5OE6h7WOt2I6ehPlVsDGEtKClOzhvDOMYtpYt6zh2TZKLzaYnX83PrP7WWBE48 Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:58:43.369733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-TestVolumeBootPattern-40689968 Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-TestVolumeBootPattern-40689968-project Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: e1012f856881485cb40d916131a5d763 Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-TestVolumeBootPattern-40689968-project Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 00:58:43.371146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '984863bd-b3a1-4546-8b36-720ba324d7ac'} Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:58:43.371997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 00:58:43.373263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:58:43.374400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Returning 404 to user: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:58:43.374400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-deb5b80c-2ab2-473f-8e5d-947ed380cf7c tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 404 len: 111 microversion: 2.1 time: 0.030565 Sep 11 00:58:43.374400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 355/697] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:43 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:58:43.480844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-252211047", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-975872804", "security_groups": [{"name": "tempest-secgroup-smoke-926362896"}], "block_device_mapping_v2": [{"uuid": "570647bd-2013-4675-8b3d-96669589b642", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "6df3e98e-548f-496e-bd19-e33fc45eb877"}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:43.620715 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] validate_networks() for [('6df3e98e-548f-496e-bd19-e33fc45eb877', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:58:44.050300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:44.051539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:44.059305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:44.059802 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:44.060230 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:44.061266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:44.061944 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:44.062718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:44.218919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:58:44.220434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:58:44.221110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:58:44.221782 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:58:44.222388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:58:44.238916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:44.239409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:44.239843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:44.241099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:44.241556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:44.242751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:44.251738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:44.262411 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for project a1368870ddbd49be857e9ac128bc684e. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:58:44.263997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:44.266377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:44.266661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for user e1012f856881485cb40d916131a5d763 and project a1368870ddbd49be857e9ac128bc684e. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:58:44.269270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-007c4b56-4035-45db-87e0-680bec2ceef9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1745 microversion: 2.20 time: 0.223256 Sep 11 00:58:44.270936 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 356/698] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:44 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1745 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:58:44.334425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:58:44.337599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='570647bd-2013-4675-8b3d-96669589b642',volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:58:44.502614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3c1d954a-beb8-4ca2-9ccf-49ab0d8ee284 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.025798 Sep 11 00:58:44.503604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 343/699] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:58:43 2022] POST /compute/v2.1/servers => generated 405 bytes in 1027 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:58:44.514753 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b50c2796-2712-4930-8521-375b7d8cdfbd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:44.515545 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b50c2796-2712-4930-8521-375b7d8cdfbd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:44.602421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b50c2796-2712-4930-8521-375b7d8cdfbd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:44.609825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b50c2796-2712-4930-8521-375b7d8cdfbd tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1093 microversion: 2.1 time: 0.099148 Sep 11 00:58:44.610392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 357/700] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:44 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1093 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:45.298178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:45.298777 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:45.305706 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:45.306899 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:45.307698 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:45.308508 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:45.309096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:45.309488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:45.484188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:45.495251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:45.496807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:45.498038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3dfd6565-75e3-4008-bead-46d852d70c6c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.205285 Sep 11 00:58:45.498509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 344/701] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:45 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:45.634609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:45.635685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:45.643106 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:45.643929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:45.644255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:45.645180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:45.645813 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:45.646242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:45.737703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:45.749034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5469eced-adb0-46ea-85d1-e73c0c813a6a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1191 microversion: 2.1 time: 0.119319 Sep 11 00:58:45.752079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 358/702] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:45 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1191 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:46.524450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:46.525228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:46.535086 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:46.535596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:46.536220 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:46.536766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:46.537299 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:46.537666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:46.559260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-b64a96a7-2595-4769-9060-108d80529b76 req-72299e4e-1dcd-4073-a87a-7306a5c7f942 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "name": "network-vif-plugged", "status": "completed", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:46.578251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-b64a96a7-2595-4769-9060-108d80529b76 req-72299e4e-1dcd-4073-a87a-7306a5c7f942 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:46.578739 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-b64a96a7-2595-4769-9060-108d80529b76 req-72299e4e-1dcd-4073-a87a-7306a5c7f942 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:46.579406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-b64a96a7-2595-4769-9060-108d80529b76 req-72299e4e-1dcd-4073-a87a-7306a5c7f942 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:46.629838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-b64a96a7-2595-4769-9060-108d80529b76 req-72299e4e-1dcd-4073-a87a-7306a5c7f942 service nova] Creating event network-vif-plugged:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 00:58:46.643697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-b64a96a7-2595-4769-9060-108d80529b76 req-72299e4e-1dcd-4073-a87a-7306a5c7f942 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089754 Sep 11 00:58:46.644933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 359/703] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:46.773285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:46.774377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:46.784834 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:46.785718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:46.788263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:46.790068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:46.790773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:46.796988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:47.194871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:47.206877 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-17c3a5fc-34a3-4a7b-87fb-c8ad11459bf0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1209 microversion: 2.1 time: 0.437600 Sep 11 00:58:47.220478 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 360/704] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:46 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1209 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:47.307966 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:47.317685 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:47.322385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:47.324335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a4d8eab3-ad29-4bb8-9533-460e428bdb11 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.804231 Sep 11 00:58:47.350230 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 345/705] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:46 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:47.439856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:47.440354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:47.451553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:47.452057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:47.452495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:47.453201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:47.453657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:47.454079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:47.471643 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:47.541838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8305a185-b33a-45fc-8f94-676e19b0d348 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/ips" status: 200 len: 150 microversion: 2.1 time: 0.105713 Sep 11 00:58:47.548241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 361/706] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:58:47 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/ips => generated 150 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:48.236422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:48.237056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:48.243866 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:48.244369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:48.244804 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:48.247052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:48.251085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:48.251542 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:48.359325 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:48.360795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:48.368891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:48.369639 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:48.370085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:48.370985 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:48.371615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:48.372045 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:48.596487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:48.605032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c3f17fa1-e8f6-4b2f-92df-fc688ef298fb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.373041 Sep 11 00:58:48.619436 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 346/707] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:48 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:48.639771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:48.653421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:48.655370 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:48.658126 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f9797265-1d94-40c8-9f48-d057e090ffff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.304276 Sep 11 00:58:48.660310 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1481813502", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "networks": [{"uuid": "e7aac99f-395b-41fd-b96e-56bde33609b0"}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:48.662810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 362/708] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:48 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:48.743401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] validate_networks() for [('e7aac99f-395b-41fd-b96e-56bde33609b0', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 00:58:48.774500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-de4a5b2e-daef-4489-ba13-ed204ad069cf req-1039e09d-ba5f-42c4-995c-36c80ebee0cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "459cb8ce-dc67-4fd5-a7ba-23b228fd415e", "tag": "bc9b80b7-364d-4952-97b3-277a78d5fa01"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:48.785494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-de4a5b2e-daef-4489-ba13-ed204ad069cf req-1039e09d-ba5f-42c4-995c-36c80ebee0cc service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:48.786238 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-de4a5b2e-daef-4489-ba13-ed204ad069cf req-1039e09d-ba5f-42c4-995c-36c80ebee0cc service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:48.786668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-de4a5b2e-daef-4489-ba13-ed204ad069cf req-1039e09d-ba5f-42c4-995c-36c80ebee0cc service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:48.799048 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-de4a5b2e-daef-4489-ba13-ed204ad069cf req-1039e09d-ba5f-42c4-995c-36c80ebee0cc service nova] Creating event network-changed:bc9b80b7-364d-4952-97b3-277a78d5fa01 for instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e on hv-cinder-maste Sep 11 00:58:48.804014 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-de4a5b2e-daef-4489-ba13-ed204ad069cf req-1039e09d-ba5f-42c4-995c-36c80ebee0cc service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034038 Sep 11 00:58:48.805021 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 363/709] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:49.137553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c5910967-d51f-45a8-bf08-f28820c2270a"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:49.140196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:49.147332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.148174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.149081 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.150003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.150731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.151394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.386617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 00:58:49.387134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 00:58:49.387556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 00:58:49.387962 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 00:58:49.388371 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 00:58:49.407455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.408214 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.408827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.409705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.410343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.410987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.430670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Getting quotas for project de36b740b8234aca8317d5e4440decec. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 00:58:49.436404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Getting quotas for user 78a2199d3f1d4bc5b1f988ae06c0d609 and project de36b740b8234aca8317d5e4440decec. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 00:58:49.454641 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 00:58:49.457224 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 00:58:49.520745 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-43c12818-6b1c-4173-a0f4-68fac86fe6bc tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.864691 Sep 11 00:58:49.531772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 347/710] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 00:58:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 866 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 00:58:49.536052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9a113f2f-f478-4ed2-9e90-f269430c62c9 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:49.536818 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9a113f2f-f478-4ed2-9e90-f269430c62c9 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:49.557956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7c3008f8-dcb3-4b0c-a7c7-ef3eff2f4009 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "POST /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.424544 Sep 11 00:58:49.566593 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 364/711] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:58:49 2022] POST /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments => generated 194 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:58:49.630740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:49.631863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:49.649546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.650458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.651296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.652237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.652700 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.653124 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.668362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9a113f2f-f478-4ed2-9e90-f269430c62c9 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:49.676338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9a113f2f-f478-4ed2-9e90-f269430c62c9 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1219 microversion: 2.1 time: 0.146793 Sep 11 00:58:49.677270 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 348/712] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:49 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1219 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:49.713758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:49.715671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:49.727484 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.728259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.728985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.729921 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:49.730837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:49.731684 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:49.997183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:50.006465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6be792f4-d32a-4ad6-b539-64bf91e071c5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.380623 Sep 11 00:58:50.007923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 365/713] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:49 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:50.071765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:50.080897 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:50.086810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:50.090442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9ee3258b-88d2-46d6-a66c-b34834eef29c tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.388452 Sep 11 00:58:50.095618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 349/714] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:49 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:50.704321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:50.706491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:50.722118 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:50.723314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:50.724515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:50.725470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:50.726157 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:50.726954 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:50.845499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:50.855124 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6c158db3-3ae9-42c6-bc2e-b18f1a0c78c4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1271 microversion: 2.1 time: 0.155917 Sep 11 00:58:50.880017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 366/715] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:50 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1271 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:51.031615 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:51.033253 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:51.040819 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:51.041655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:51.042314 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:51.043264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:51.044232 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:51.044769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:51.123146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:51.124064 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:51.131756 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:51.133439 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:51.134190 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:51.135176 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:51.136004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:51.136825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:51.285451 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:51.297383 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2680325d-ed48-4f02-ac20-870ae2c3d94b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.270912 Sep 11 00:58:51.307487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 350/716] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:51 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:51.441556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:51.455578 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:51.457016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:51.458212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f67ffc7b-77fe-4aa8-8fb7-bd3582e294a1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.340504 Sep 11 00:58:51.458778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 367/717] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:51 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:51.888862 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:51.889487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:51.896266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:51.897172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:51.898159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:51.899228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:51.899620 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:51.900080 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:51.921827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:51.990681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:52.005404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-92dd2513-7b4e-477b-93b8-b55e37c20d90 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1277 microversion: 2.1 time: 0.121384 Sep 11 00:58:52.006824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 351/718] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:51 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1277 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:52.337630 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:52.338440 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:52.347858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:52.348344 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:52.348786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:52.349512 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:52.349971 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:52.350399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:52.489919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:52.491426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:52.498703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:52.500319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:52.501186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:52.502234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:52.502951 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:52.503565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:52.727315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:52.736693 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-86cda478-dd11-4e9e-97ea-32737568f246 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.403656 Sep 11 00:58:52.754421 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 368/719] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:52 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:52.832916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:52.844024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:52.845744 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:52.847522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-598958c8-70b2-4b97-aa81-4e38d8bedc3b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.361793 Sep 11 00:58:52.848334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 352/720] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:52 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:58:53.030219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:53.030863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:53.037672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.039057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.039495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:53.040445 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.040913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.041349 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:53.285229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:53.308101 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5d2033f1-c61b-40a9-850f-ceb346b2fede tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.281612 Sep 11 00:58:53.308894 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 369/721] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:53 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:53.441074 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-6c3b349a-9724-4e35-ade2-b77d3ab48658 req-5e1355ea-a401-4685-8b3a-f8a3959d297c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "28ecae2d-9fbd-42c1-932c-40406ca730b0", "tag": "1717a3c3-263a-47c4-85ce-30d613700835"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:53.459779 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6c3b349a-9724-4e35-ade2-b77d3ab48658 req-5e1355ea-a401-4685-8b3a-f8a3959d297c service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.462228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6c3b349a-9724-4e35-ade2-b77d3ab48658 req-5e1355ea-a401-4685-8b3a-f8a3959d297c service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.462952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6c3b349a-9724-4e35-ade2-b77d3ab48658 req-5e1355ea-a401-4685-8b3a-f8a3959d297c service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:53.475056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-6c3b349a-9724-4e35-ade2-b77d3ab48658 req-5e1355ea-a401-4685-8b3a-f8a3959d297c service nova] Creating event network-changed:1717a3c3-263a-47c4-85ce-30d613700835 for instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 on hv-cinder-maste Sep 11 00:58:53.496329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-6c3b349a-9724-4e35-ade2-b77d3ab48658 req-5e1355ea-a401-4685-8b3a-f8a3959d297c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059208 Sep 11 00:58:53.497254 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 353/722] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:53.763814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:53.765324 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:53.774456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.775273 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.775929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:53.776869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.777487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.778072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:53.874871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:53.876029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:53.887440 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.888325 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.889032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:53.889984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:53.890832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:53.891836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:54.370913 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:54.376569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-eda27843-3cf2-4860-812c-6dc949db1cef tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.618340 Sep 11 00:58:54.384749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:54.390372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 370/723] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:53 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:54.395102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:54.395974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:54.403068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:54.403710 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:54.404386 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:54.405093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:54.405725 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:54.406193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:54.418548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:54.420246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:54.421528 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ec8d2dfd-3ebe-427b-a3d8-9bcf41dd74c1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.551549 Sep 11 00:58:54.425534 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 354/724] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:53 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:54.590500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:54.599984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cf9b9474-d35d-442b-aff1-210517749848 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.207428 Sep 11 00:58:54.599984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 371/725] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:54 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:55.416767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:55.417392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:55.424817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:55.425562 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:55.426159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:55.427059 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:55.427700 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:55.428151 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:55.449973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:55.450755 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:55.454085 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:55.458695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:55.459226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:55.459660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:55.460638 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:55.461311 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:55.461741 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:55.689402 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:55.703241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:55.707594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-adf57183-f997-4203-9a7c-05060109bdce tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.288186 Sep 11 00:58:55.713720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:55.714109 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 355/726] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:55 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:55.727183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:55.730947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e4c1abff-592c-41af-950b-6ab38b0530af tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.282861 Sep 11 00:58:55.732525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:55.733046 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 372/727] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:55 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:55.733703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:55.745449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:55.746570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:55.747554 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:55.748741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:55.749499 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:55.750107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:55.899730 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:55.908269 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c3504b10-aaea-42d9-9eb7-987c27ba216f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.184414 Sep 11 00:58:55.908856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 356/728] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:55 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:56.737653 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:56.738960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:56.746531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:56.747123 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:56.747805 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:56.748500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:56.748949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:56.749364 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:56.759166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:56.760511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:56.767056 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:56.767931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:56.768613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:56.769530 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:56.770264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:56.770944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:56.993342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:57.007176 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:57.008778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:57.010140 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-60419793-2cae-4ad0-bae1-fc0eccf9c4b9 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1740 microversion: 2.20 time: 0.256463 Sep 11 00:58:57.010690 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 357/729] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:56 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1740 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:58:57.031853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:57.032740 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:57.036650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:57.044932 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:57.045425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:57.045864 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:57.046573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:57.047063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:57.047497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:57.048996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-02e3e17f-38b6-4fbf-85a3-43c0344e4be9 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.315070 Sep 11 00:58:57.049767 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 373/730] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:56 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:57.275326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:57.286348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8d964701-a58d-4cb0-87ce-48bb4410ff33 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.274362 Sep 11 00:58:57.286982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 358/731] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:57 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:57.395556 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:57.396052 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:57.402559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:57.403059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:57.403492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:57.404171 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:57.404621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:57.405044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:57.483746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'services' on Instance uuid fa6184df-9b60-42df-b8af-547204e85659 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:58:57.558034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a3887dd2-dfe2-40e8-b66f-cc27c43b7332 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments/c5910967-d51f-45a8-bf08-f28820c2270a" status: 202 len: 0 microversion: 2.1 time: 0.166140 Sep 11 00:58:57.558700 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 374/732] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:58:57 2022] DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments/c5910967-d51f-45a8-bf08-f28820c2270a => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:58:57.567263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:57.570298 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:57.576772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:57.577268 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:57.577713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:57.578405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:57.578889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:57.579318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:57.608833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-89b98265-7598-41bb-a300-503b45b5c67a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045598 Sep 11 00:58:57.609379 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 359/733] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:58:57 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:58.037095 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:58.037718 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:58.044441 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.044927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.045363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.046055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.046507 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.046964 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.072670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:58.073954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:58.080345 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.080828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.081308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.082007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.082456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.082899 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.334371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:58.345648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:58.347408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:58.348625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1ae3af9d-3633-4a43-9278-b3d002884fad tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1722 microversion: 2.20 time: 0.315988 Sep 11 00:58:58.349599 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 375/734] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:58 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1722 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:58.354557 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:58.354750 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:58.355390 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:58.358561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=66 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:58.363540 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.364271 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.365037 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.366819 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.367952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.368626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.370300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f7b8a512-f8a2-47a7-8071-8ee98138d877 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.302217 Sep 11 00:58:58.371367 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 360/735] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:58 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:58.590826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:58.595866 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:58.598187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-18ec90e5-c3b4-4c5e-bc45-f38812df49af tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.247604 Sep 11 00:58:58.613760 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 376/736] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:58 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:58.622599 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:58.622739 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-1e6af349-5352-4c8e-bf30-14eb3cbe7e7f req-a33e0793-b519-4009-b02c-f9630bd2ace6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "28ecae2d-9fbd-42c1-932c-40406ca730b0", "name": "network-vif-plugged", "status": "completed", "tag": "1717a3c3-263a-47c4-85ce-30d613700835"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:58:58.625022 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:58.634402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.634402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.634402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.635110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.635751 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.636454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.639359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-1e6af349-5352-4c8e-bf30-14eb3cbe7e7f req-a33e0793-b519-4009-b02c-f9630bd2ace6 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.639921 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-1e6af349-5352-4c8e-bf30-14eb3cbe7e7f req-a33e0793-b519-4009-b02c-f9630bd2ace6 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.640638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-1e6af349-5352-4c8e-bf30-14eb3cbe7e7f req-a33e0793-b519-4009-b02c-f9630bd2ace6 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.655714 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-1e6af349-5352-4c8e-bf30-14eb3cbe7e7f req-a33e0793-b519-4009-b02c-f9630bd2ace6 service nova] Creating event network-vif-plugged:1717a3c3-263a-47c4-85ce-30d613700835 for instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 on hv-cinder-maste Sep 11 00:58:58.666619 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-1e6af349-5352-4c8e-bf30-14eb3cbe7e7f req-a33e0793-b519-4009-b02c-f9630bd2ace6 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060620 Sep 11 00:58:58.667787 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 361/737] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:58:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:58:58.675053 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-11add6bc-714d-4349-8d48-85b496d4f07d tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.056846 Sep 11 00:58:58.675853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 377/738] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 00:58:58 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 11 00:58:58.759703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:58.760576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:58.767284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.767856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.768303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.769002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.769459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.769891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.797140 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:58:58.873801 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b34cfd5a-4556-4c0e-8253-707bdf3b18c1 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 204 len: 0 microversion: 2.1 time: 0.122571 Sep 11 00:58:58.874370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 362/739] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:58:58 2022] DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 00:58:58.883183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:58.883803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:58.889693 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.890181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.890617 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:58.891339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:58.891795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:58.892222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.026408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:59.034739 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-91bb23ea-af1b-4994-80a9-589aa50a307a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1680 microversion: 2.1 time: 0.156056 Sep 11 00:58:59.035354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 378/740] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:58 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:58:59.379373 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:59.379982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:59.382987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:INSERT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:59.389002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:59.389492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:59.390026 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.390746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:59.391226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:59.391646 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.394724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:59.395370 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:59.402694 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:59.403212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:59.403647 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.404336 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:59.404786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:59.405203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.426219 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=4 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:59.586818 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:59.595077 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:59.595749 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:59.596567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:59.597746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-01a217e0-b09f-48cc-b107-1e2c35cd2e45 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1722 microversion: 2.20 time: 0.222771 Sep 11 00:58:59.598303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 363/741] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:58:59 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1722 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:58:59.604223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6c25888e-0272-4967-ac59-20bfdcd905f1 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.213242 Sep 11 00:58:59.604781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 379/742] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:59 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:58:59.621360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:58:59.621974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:58:59.627917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:59.628409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:59.628849 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.629592 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:58:59.629997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:58:59.630423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:58:59.672778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=14 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:59.759090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:58:59.766661 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2917d34a-f588-4611-b3a2-83d08039243d tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.149115 Sep 11 00:58:59.767251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 364/743] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:58:59 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:00.058393 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:00.059028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:00.064911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.065389 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.065820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.066499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.066972 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.067388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.201691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:00.209402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-21d75c52-cc7f-4f71-9ba7-1a3d013cb307 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1680 microversion: 2.1 time: 0.154977 Sep 11 00:59:00.210577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 380/744] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:00 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:00.623546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:00.624369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:00.627547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:00.628107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:00.631158 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.631408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.632664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.633550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.634005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.634426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.635263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.635752 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.636186 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.636883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.637339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.637765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.823949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:00.831983 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:00.832274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:00.833426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:00.834605 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-30b4df26-76ef-4385-a427-55e75ef5537f tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1722 microversion: 2.20 time: 0.215053 Sep 11 00:59:00.835377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 365/745] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:00 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1722 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:00.852982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "459cb8ce-dc67-4fd5-a7ba-23b228fd415e", "name": "network-vif-plugged", "status": "completed", "tag": "bc9b80b7-364d-4952-97b3-277a78d5fa01"}, {"server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "name": "network-vif-unplugged", "status": "completed", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:00.853166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8c2cbfc5-f980-444d-abbf-b15ba6c64191 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.216651 Sep 11 00:59:00.853166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 381/746] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:00 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:59:00.853166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:00.853166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:00.853488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.853488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.853488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.853735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.853735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.853735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.853987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:00.854360 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:00.854809 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:00.864905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] Creating event network-vif-plugged:bc9b80b7-364d-4952-97b3-277a78d5fa01 for instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e on hv-cinder-maste Sep 11 00:59:00.865416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] Creating event network-vif-unplugged:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 00:59:00.872724 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-54026e54-dea9-4bb2-86cc-87128e6759bb req-e7161052-e495-488b-9ee8-4ad45a9899c0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.036098 Sep 11 00:59:00.873314 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 366/747] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:00 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:00.986771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:00.995379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7d94c2e2-b21b-47ea-bbba-19b68f4a1b4c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.153552 Sep 11 00:59:00.995938 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 382/748] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:00 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:01.233738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:01.234444 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:01.240641 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:01.241455 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:01.241891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:01.242585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:01.243072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:01.243511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:01.373867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:01.382302 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-937d16b8-84e5-4c03-a3ae-9f6083e3b360 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1680 microversion: 2.1 time: 0.152073 Sep 11 00:59:01.383129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 367/749] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:01 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:01.861641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:01.862314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:01.865566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:01.867466 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:01.869890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:01.871049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:01.871446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:01.872227 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:01.872721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:01.873145 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:01.874587 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:01.875096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:01.875533 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:01.876249 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:01.876701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:01.877120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:01.894826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:02.104251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:02.112613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:02.114133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:02.115392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ea7108c7-6640-40c2-bfa1-540dc65201da tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1722 microversion: 2.20 time: 0.258438 Sep 11 00:59:02.116648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 383/750] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:01 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1722 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:59:02.121326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:02.121728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:02.122548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:02.128714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:02.129467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:02.129960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:02.130435 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1fa68e96-824c-46d4-aeed-87f6fce86d28 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.269724 Sep 11 00:59:02.130878 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:02.131338 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 368/751] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:01 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:59:02.131487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:02.132730 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:02.268547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:02.276687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2aa6caa2-02d8-4c29-9a5c-4ed2b86216e4 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1319 microversion: 2.1 time: 0.158652 Sep 11 00:59:02.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 384/752] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:02 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:02.407520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:02.408147 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:02.423130 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:02.424041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:02.424561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:02.425288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:02.425760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:02.426194 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:02.574681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:02.583245 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-32523776-c366-47f0-a7f0-2a3720aac30b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1680 microversion: 2.1 time: 0.181575 Sep 11 00:59:02.583813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 369/753] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:02 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1680 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:03.147272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:03.147961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:03.153611 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:03.154685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.154923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:03.155205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.155834 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.156538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.157287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.157709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.161828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.162324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.163334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.164044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.164501 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.164931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.447201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:03.455435 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:03.456882 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:03.458050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-eb2ef63b-fcff-4876-942b-28d4a2e48dff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1725 microversion: 2.20 time: 0.314890 Sep 11 00:59:03.458612 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 385/754] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:03 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1725 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:03.463471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:03.464086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:03.470005 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.470501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.470989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.471686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.472140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.472565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.472674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:03.488982 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-14e5f187-8b53-4ff3-b6b4-2b90d39dde62 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1264 microversion: 2.1 time: 0.339363 Sep 11 00:59:03.489565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 370/755] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:03 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1264 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:59:03.495231 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a0a5a982-f050-4611-934e-6b7bf026b8e7"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:03.498473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:03.505265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.505763 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.506346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.507153 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.507612 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.508041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.829617 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-892e49b2-61a3-4ca7-99df-8a18588e8f19 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.338666 Sep 11 00:59:03.830585 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 371/756] 10.222.0.27 () {62 vars in 1364 bytes} [Sun Sep 11 00:59:03 2022] POST /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-volume_attachments => generated 188 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 11 00:59:03.841335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:03.842727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:03.848720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.849337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.849872 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.851253 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.851816 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.852508 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.960721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:03.968379 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6bafa5ad-f1e9-4f40-82a4-5107972eec94 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1613 microversion: 2.1 time: 0.131137 Sep 11 00:59:03.969233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 372/757] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:03 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1613 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:03.974743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:03.976226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:03.981525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.982014 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.982497 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.983156 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:03.983609 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:03.984042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:03.985672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:03.997533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4b9d3b6e-7c74-4f02-a3ad-bf55a30398c1 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1508 microversion: 2.1 time: 0.537043 Sep 11 00:59:04.008068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 386/758] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:03 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1508 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:04.070902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-8cd2d60b-8a01-430c-8563-950a60d467b6 req-27d1dc6a-e3ec-457c-8324-ea247744650a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa6184df-9b60-42df-b8af-547204e85659", "name": "network-vif-deleted", "tag": "8ecae1b4-4f80-425a-a9d0-bcd2ab86597f"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:04.102452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-8cd2d60b-8a01-430c-8563-950a60d467b6 req-27d1dc6a-e3ec-457c-8324-ea247744650a service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.103044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-8cd2d60b-8a01-430c-8563-950a60d467b6 req-27d1dc6a-e3ec-457c-8324-ea247744650a service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.103549 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-8cd2d60b-8a01-430c-8563-950a60d467b6 req-27d1dc6a-e3ec-457c-8324-ea247744650a service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.118487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-8cd2d60b-8a01-430c-8563-950a60d467b6 req-27d1dc6a-e3ec-457c-8324-ea247744650a service nova] Creating event network-vif-deleted:8ecae1b4-4f80-425a-a9d0-bcd2ab86597f for instance fa6184df-9b60-42df-b8af-547204e85659 on hv-cinder-maste Sep 11 00:59:04.124096 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-8cd2d60b-8a01-430c-8563-950a60d467b6 req-27d1dc6a-e3ec-457c-8324-ea247744650a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057359 Sep 11 00:59:04.141850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 387/759] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:04.307200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cf0d565f-6809-4340-be6f-afccd7665fa0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-volume_attachments/a0a5a982-f050-4611-934e-6b7bf026b8e7" status: 202 len: 0 microversion: 2.20 time: 0.336011 Sep 11 00:59:04.308740 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 373/760] 10.222.0.27 () {60 vars in 1457 bytes} [Sun Sep 11 00:59:03 2022] DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-volume_attachments/a0a5a982-f050-4611-934e-6b7bf026b8e7 => generated 0 bytes in 338 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 11 00:59:04.382245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:04.383582 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:04.392324 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.392815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.393384 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.394306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.394763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.395214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.513501 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:04.514146 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:04.519853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=126981) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 11 00:59:04.524923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.525423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.525861 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.526566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.527043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.527463 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.645070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5b6f0b6b-f4ba-480f-a3b7-92392b41aadc tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "POST /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/action" status: 202 len: 0 microversion: 2.20 time: 0.270344 Sep 11 00:59:04.645934 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 388/761] 10.222.0.27 () {62 vars in 1319 bytes} [Sun Sep 11 00:59:04 2022] POST /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/action => generated 0 bytes in 274 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:59:04.658763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:04.659897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:04.669275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.669781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.670215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.671012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.671517 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.671943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.771090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:04.783722 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cc345812-c33c-45d7-9afd-d22bc0435acf tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1450 microversion: 2.1 time: 0.273965 Sep 11 00:59:04.784303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 374/762] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:04 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1450 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:04.809360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:04.810193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:04.816545 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.817345 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.818018 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:04.819588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:04.820221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:04.820808 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:05.043635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:05.060580 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:05.065161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:05.066890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e83c3d8b-0303-409b-9b11-6299333ce3ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1733 microversion: 2.20 time: 0.413972 Sep 11 00:59:05.067579 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 389/763] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:04 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1733 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:05.074558 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:05.075492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:05.099434 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:05.101781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:05.102357 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:05.103138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:05.103603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:05.104031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:05.166845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:05.167284 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:05.167719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:05.168205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lazy-loading 'fault' on Instance uuid fa6184df-9b60-42df-b8af-547204e85659 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:59:05.172161 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:05.181246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b629768e-5279-445b-be8d-d2b52521d0b6 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1450 microversion: 2.1 time: 0.378543 Sep 11 00:59:05.181883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 375/764] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:04 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1450 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 00:59:05.217801 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd886b168a8eb4424987555c073d71727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172e557567404e0f906fd1c3452148d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:05.227958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e9a65bdd-27c6-4a65-a8ed-597ae72003f5 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 200 len: 1330 microversion: 2.1 time: 0.158860 Sep 11 00:59:05.229350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 390/765] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:05 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 1330 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:06.094096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:06.094709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:06.101581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:06.102113 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:06.102732 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:06.103474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:06.103923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:06.104345 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:06.252823 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:06.254109 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:06.254650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:06.264767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:06.266766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:06.267468 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:06.268161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:06.268596 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c275aebf-02f9-4536-9e55-94f24eb88809 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.178344 Sep 11 00:59:06.268952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:06.269400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 376/766] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:06 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:06.270027 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:06.270578 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:06.271193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:06.291315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 HTTP/1.1 Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:06.293840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:06.295505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'fa6184df-9b60-42df-b8af-547204e85659'} Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:06.296459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:06.297991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:06.300868 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:06.302097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-54708d50-57d0-4ba8-9735-7bf707e9c9b7 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 404 len: 111 microversion: 2.1 time: 0.053136 Sep 11 00:59:06.307577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 391/767] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:06 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:06.313172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0e0f9d40-677c-4888-99a9-9bf2e73aec71 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '.version_select at 0x7fc01aa7bc10>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:06.333307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0e0f9d40-677c-4888-99a9-9bf2e73aec71 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-308857369" status: 202 len: 0 microversion: 2.1 time: 0.024154 Sep 11 00:59:06.333953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 377/768] 10.222.0.27 () {58 vars in 1218 bytes} [Sun Sep 11 00:59:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-308857369 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:59:07.295433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:07.297251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:07.308176 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:07.308659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:07.309278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:07.309983 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:07.310427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:07.310876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:07.515197 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:07.527741 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:07.529219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:07.536633 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-af0230b6-5000-4dfd-89a5-27baecf787f0 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.245084 Sep 11 00:59:07.538407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 392/769] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:07 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:08.119063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-b9aa71a3-f365-4106-8d2c-ce9792e85e91 req-7397b121-d609-4a71-ba7d-e3aa6710def3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "459cb8ce-dc67-4fd5-a7ba-23b228fd415e", "tag": "bc9b80b7-364d-4952-97b3-277a78d5fa01"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:08.143348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-b9aa71a3-f365-4106-8d2c-ce9792e85e91 req-7397b121-d609-4a71-ba7d-e3aa6710def3 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.144106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-b9aa71a3-f365-4106-8d2c-ce9792e85e91 req-7397b121-d609-4a71-ba7d-e3aa6710def3 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.144728 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-b9aa71a3-f365-4106-8d2c-ce9792e85e91 req-7397b121-d609-4a71-ba7d-e3aa6710def3 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.170204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-b9aa71a3-f365-4106-8d2c-ce9792e85e91 req-7397b121-d609-4a71-ba7d-e3aa6710def3 service nova] Creating event network-changed:bc9b80b7-364d-4952-97b3-277a78d5fa01 for instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e on hv-cinder-maste Sep 11 00:59:08.184108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-b9aa71a3-f365-4106-8d2c-ce9792e85e91 req-7397b121-d609-4a71-ba7d-e3aa6710def3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.301087 Sep 11 00:59:08.185065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 378/770] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:08.580791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:08.581418 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:08.588631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.589476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.590115 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.591226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.591925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.592606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.629845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:08.763385 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:08.773910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:08.775603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:08.776784 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e360c6ea-c4f6-4132-ab8d-e834ed82cc09 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.200525 Sep 11 00:59:08.781229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 393/771] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:08 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:08.833456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:08.834073 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:08.867309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.867907 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.868369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.869119 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.869580 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.870014 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.885766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 HTTP/1.1 Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:08.887510 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.889339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'fa6184df-9b60-42df-b8af-547204e85659'} Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.891944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.894261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:08.894261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-890c0b42-cee0-4e1c-9a23-f4822bf2c71c tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 404 len: 111 microversion: 2.1 time: 0.059846 Sep 11 00:59:08.894261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 379/772] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:59:08 2022] DELETE /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:08.899730 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:08.900742 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fa6184df-9b60-42df-b8af-547204e85659] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:08.914183 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.914907 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.915444 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.916277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:08.916866 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:08.917391 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:08.942069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 HTTP/1.1 Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:08.944867 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.946585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'fa6184df-9b60-42df-b8af-547204e85659'} Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:08.947514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. Sep 11 00:59:08.948958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:08.950296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance fa6184df-9b60-42df-b8af-547204e85659 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:08.950663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2b373ba4-39de-4953-a88d-bf4de2ba8b1a tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659" status: 404 len: 111 microversion: 2.1 time: 0.055585 Sep 11 00:59:08.951278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 394/773] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:08 2022] GET /compute/v2.1/servers/fa6184df-9b60-42df-b8af-547204e85659 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 00:59:09.805227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:09.806482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:09.812760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:09.813581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:09.814261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:09.815089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:09.815636 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:09.816154 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:09.976091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:09.988221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:09.989689 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:09.991714 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6d1bace0-3321-44bc-b1eb-2780b6105c54 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.190471 Sep 11 00:59:09.992768 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 380/774] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:09 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:10.155034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:10.155546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:10.167436 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.167973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.168432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.169232 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.169621 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.170093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.188791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 HTTP/1.1 Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:10.190634 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.192268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650'} Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:10.193184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.195235 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.196728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:10.198003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-86e9304e-716e-4bde-856a-1696cd526161 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 404 len: 111 microversion: 2.1 time: 0.046872 Sep 11 00:59:10.198956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 395/775] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:59:10 2022] DELETE /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:10.206750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:10.207453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:10.214847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.215349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.215787 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.216494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.216952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.217377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.232680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 HTTP/1.1 Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:10.234258 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.235846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650'} Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:10.236840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. Sep 11 00:59:10.238380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.239808 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:10.239808 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ecd3446b-a76e-4756-869f-3a7decd145a0 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650" status: 404 len: 111 microversion: 2.1 time: 0.032846 Sep 11 00:59:10.239808 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 381/776] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:10 2022] GET /compute/v2.1/servers/b9686dd4-5cbd-4bd2-b41f-6c1dd08ff650 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:10.244520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:10.245026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:10.251682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.252182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.252936 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.253822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.254624 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.255463 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.274246 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 HTTP/1.1 Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:10.275952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.277275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '9a843e04-0782-4e1e-b379-9be9a7106442'} Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:10.278013 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.279649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.282036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:10.282036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-600ec26c-d4d2-4bdd-bc91-4cd85f55392e tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 404 len: 111 microversion: 2.1 time: 0.036768 Sep 11 00:59:10.282036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 396/777] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:59:10 2022] DELETE /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:10.287699 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:10.288415 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: 9a843e04-0782-4e1e-b379-9be9a7106442] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:10.296030 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.297003 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.297117 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.297934 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:10.298487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:10.299041 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:10.321840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 HTTP/1.1 Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:10.323425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.324827 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '9a843e04-0782-4e1e-b379-9be9a7106442'} Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:10.325626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. Sep 11 00:59:10.326912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:10.328022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance 9a843e04-0782-4e1e-b379-9be9a7106442 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:10.328022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e749af55-ac18-4f35-b1c6-9c565caef301 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442" status: 404 len: 111 microversion: 2.1 time: 0.041401 Sep 11 00:59:10.328980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 382/778] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:10 2022] GET /compute/v2.1/servers/9a843e04-0782-4e1e-b379-9be9a7106442 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:11.027876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:11.028484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:11.035035 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:11.035497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:11.036074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:11.037262 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:11.037704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:11.039694 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:11.195856 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:11.205043 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:11.206890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:11.208365 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-498c796b-2e89-4eee-907d-5242b2b3035b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.184493 Sep 11 00:59:11.209619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 397/779] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:11 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:11.486909 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:11.489356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:11.497543 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:11.498326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:11.498989 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:11.499894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:11.500366 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:11.500784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:11.515529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf HTTP/1.1 Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:11.517227 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.519512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'fb960ab4-4a34-4074-813f-cd971d916bcf'} Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:11.521008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.523347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.525547 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:11.525547 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cdcb58fa-dbfd-47d1-ba1f-170f56801e2b tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 404 len: 111 microversion: 2.1 time: 0.035928 Sep 11 00:59:11.533137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:11.533753 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] [instance: fb960ab4-4a34-4074-813f-cd971d916bcf] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:11.538895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 383/780] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:59:11 2022] DELETE /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:11.543513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:11.544125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:11.544591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:11.545510 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:11.546430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:11.546736 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:11.564925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] HTTP exception thrown: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Request method failure captured: Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf HTTP/1.1 Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFslUIznaorsPIZaGid_BVcz2tS7msliGJO6llbR1zgJT2LL3q1MEeEBtpCUhbKsZrljkGzJjkKlsXVR4NQ_EG8ZAI6EATJXyL224nTSXEFLXjI2ofMgpJiMqSh_DcmilITIu6jJ9MgrLgMjROf2Io8GVz0BKkgw4ZolOwHgwy_uUk Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:11.567348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/172e557567404e0f906fd1c3452148d8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/172e557567404e0f906fd1c3452148d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_172e557567404e0f906fd1c3452148d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 172e557567404e0f906fd1c3452148d8 Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-817797633 Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: d886b168a8eb4424987555c073d71727 Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeNegativeTest-817797633-project Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.569447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'fb960ab4-4a34-4074-813f-cd971d916bcf'} Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:11.570223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. Sep 11 00:59:11.571464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:11.572629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] Returning 404 to user: Instance fb960ab4-4a34-4074-813f-cd971d916bcf could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:11.572629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8b52c695-f040-49bb-b5b5-9da2e05a0c31 tempest-AttachVolumeNegativeTest-817797633 tempest-AttachVolumeNegativeTest-817797633-project] 10.222.0.27 "GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf" status: 404 len: 111 microversion: 2.1 time: 0.040285 Sep 11 00:59:11.572629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 398/781] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:11 2022] GET /compute/v2.1/servers/fb960ab4-4a34-4074-813f-cd971d916bcf => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:12.238135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:12.238754 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:12.248664 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:12.249432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:12.249865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:12.250854 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:12.251472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:12.252062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:12.269464 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=87,nova_cell1:INSERT=5,nova_cell1:UPDATE=8 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:12.405429 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:12.434515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:12.436382 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:12.437766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-db35f0ea-df49-47bc-ac43-7fafb676506e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.204022 Sep 11 00:59:12.438852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 384/782] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:12 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:13.469407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:13.470068 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:13.487510 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:13.488044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:13.488487 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:13.489214 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:13.489664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:13.490083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:13.824166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:13.833546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:13.840253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:13.848147 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6f7210e9-ff34-4de2-aa07-fc9e3ea81fb3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.389406 Sep 11 00:59:13.848659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 399/783] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:13 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:14.877001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:14.877580 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:14.885500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:14.886151 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:14.886724 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:14.887789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:14.888352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:14.888885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:15.093215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:15.106756 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:15.108406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:15.109675 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b45528ec-e7be-4f06-b09a-0674402787b7 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.237201 Sep 11 00:59:15.110334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 385/784] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:14 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:16.137514 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:16.138135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:16.144807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:16.145297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:16.145768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:16.146523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:16.147018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:16.147447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:16.290350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:16.299988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:16.301427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:16.302646 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-93e1518a-c56e-41dd-8eb0-786a2604d9b1 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.168983 Sep 11 00:59:16.303708 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 400/785] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:16 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:16.819842 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a39d4ddd-6065-4b82-a54b-99635428b70c tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Action: 'create', calling method: .version_select at 0x7fc01c6b6dc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-405204201"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:17.125298 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a39d4ddd-6065-4b82-a54b-99635428b70c tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.416676 Sep 11 00:59:17.125949 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 386/786] 10.222.0.27 () {60 vars in 1155 bytes} [Sun Sep 11 00:59:16 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 418 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 11 00:59:17.139312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2ece4682-3e36-49cb-85b9-8e1bb600308c tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01aaa2310>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:17.195276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2ece4682-3e36-49cb-85b9-8e1bb600308c tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 200 len: 344 microversion: 2.1 time: 0.059510 Sep 11 00:59:17.195970 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 401/787] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:59:17 2022] GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 344 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:17.328323 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:17.329053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:17.336213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:17.336712 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:17.337155 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:17.337888 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:17.338348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:17.338998 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:17.471257 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:17.479867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:17.481307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:17.482468 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a98bbac9-20b3-425b-9d01-6072b8927d71 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.158244 Sep 11 00:59:17.483050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 387/788] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:17 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:59:18.209906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-025d4a29-74ae-4b52-b539-cde02e59120c tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01abc9550>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:18.255891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-025d4a29-74ae-4b52-b539-cde02e59120c tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 200 len: 345 microversion: 2.1 time: 0.050023 Sep 11 00:59:18.256658 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 402/789] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:59:18 2022] GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 345 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:18.269482 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-bc4cb841-8d72-44d6-bd37-1749f59476d5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Action: 'create', calling method: .version_select at 0x7fc01a92bdc0>, body: {"snapshot": {"volume_id": "9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1293696988"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:18.428192 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-bc4cb841-8d72-44d6-bd37-1749f59476d5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.162076 Sep 11 00:59:18.435601 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 388/790] 10.222.0.27 () {60 vars in 1162 bytes} [Sun Sep 11 00:59:18 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:18.440241 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2e737605-ab87-4eb2-af5b-6051a0251c56 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01aa7a310>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:18.476660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2e737605-ab87-4eb2-af5b-6051a0251c56 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 287 microversion: 2.1 time: 0.039883 Sep 11 00:59:18.485327 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 403/791] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:18 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 287 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:59:18.492304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0034da40-8144-4191-b300-05da69d87550 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01ac27040>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:18.508975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:18.509838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:18.516465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:18.516952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:18.517531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:18.518242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:18.518906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:18.519339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:18.525673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0034da40-8144-4191-b300-05da69d87550 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.037636 Sep 11 00:59:18.526246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 389/792] 10.222.0.27 () {58 vars in 1140 bytes} [Sun Sep 11 00:59:18 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:18.537898 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0e2cb61e-8452-4137-8112-47de0272b9a5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01a960ee0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:18.582257 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0e2cb61e-8452-4137-8112-47de0272b9a5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 287 microversion: 2.1 time: 0.047987 Sep 11 00:59:18.582854 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 390/793] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:18 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 287 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:59:18.674325 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:18.682885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:18.684370 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:18.685577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1e8978f7-c3bd-4b40-8e32-40c89e863bea tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.181394 Sep 11 00:59:18.686168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 404/794] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:18 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 11 00:59:19.595651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3921e251-b526-42a2-beed-62fe9e155423 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01a92b0d0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:19.625416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3921e251-b526-42a2-beed-62fe9e155423 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 287 microversion: 2.1 time: 0.034017 Sep 11 00:59:19.625927 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 391/795] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:19 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:19.712028 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:19.712673 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:19.718988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:19.719471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:19.719901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:19.720584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:19.721035 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:19.721457 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:19.850275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:19.858394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:19.859943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:19.861089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-0d786118-8b8b-4bf5-a157-592ba2f431e5 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.153656 Sep 11 00:59:19.861641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 405/796] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:19 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:20.638680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e963d992-e4c6-4fee-8d75-0d574b039231 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01ac8f820>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:20.668667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e963d992-e4c6-4fee-8d75-0d574b039231 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 287 microversion: 2.1 time: 0.034378 Sep 11 00:59:20.669228 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 392/797] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:20 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 287 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:20.887047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:20.887670 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:20.894099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:20.894517 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:20.894973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:20.895654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:20.896099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:20.896552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:21.021894 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:21.029889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:21.031351 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:21.032765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b55ce342-4a87-4600-8026-2fcdafc7eb07 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.149642 Sep 11 00:59:21.033322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 406/798] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:20 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:21.682064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cebe495d-376e-4b7d-be9a-e0eb06f2d256 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01aae85e0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:21.711307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cebe495d-376e-4b7d-be9a-e0eb06f2d256 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 287 microversion: 2.1 time: 0.033668 Sep 11 00:59:21.711871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 393/799] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:21 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:22.058204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:22.058844 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:22.065461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:22.065943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:22.066400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:22.067107 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:22.067557 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:22.068468 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:22.198052 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:22.206470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:22.208002 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:22.210174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7b43be94-4550-49e1-acbb-8a5c5def2e61 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.155961 Sep 11 00:59:22.210802 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 407/800] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:22 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:22.725209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0281e51d-4518-428b-b7c9-168d63df8521 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01a9600d0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:22.754920 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0281e51d-4518-428b-b7c9-168d63df8521 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 288 microversion: 2.1 time: 0.034621 Sep 11 00:59:22.755517 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 394/801] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:22 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 288 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:22.766705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5fc3f517-bcfa-4a4b-97da-d54e5f1b30b2 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01aa73040>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:22.824912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5fc3f517-bcfa-4a4b-97da-d54e5f1b30b2 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "DELETE /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 202 len: 0 microversion: 2.1 time: 0.061561 Sep 11 00:59:22.825690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 408/802] 10.222.0.27 () {58 vars in 1254 bytes} [Sun Sep 11 00:59:22 2022] DELETE /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:59:22.834764 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a1de710c-65d6-4bbc-843a-8c3fe261b6ef tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01abde700>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:22.863783 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b8d2ce3d-f837-4c35-a7ad-3185ea06f91c", "device": "/dev/sdb"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:22.867477 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:22.872215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a1de710c-65d6-4bbc-843a-8c3fe261b6ef tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 200 len: 287 microversion: 2.1 time: 0.041278 Sep 11 00:59:22.872885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 395/803] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:22 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 287 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:59:22.873732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:22.874223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:22.874660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:22.875387 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:22.875842 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:22.876263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:23.138049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2dcd859f-aefd-4547-ad7a-92f7b5fbe847 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "POST /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.277874 Sep 11 00:59:23.138957 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 409/804] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 00:59:22 2022] POST /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0/os-volume_attachments => generated 194 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:59:23.235114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:23.235726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:23.242088 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:23.242569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:23.243038 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:23.243727 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:23.244177 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:23.244598 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:23.379053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:23.387116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:23.388555 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:23.389746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-606de662-b010-43ab-b3ff-a3556cc4ec27 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.158653 Sep 11 00:59:23.390330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 396/805] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:23 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:23.591343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-508070ba-924a-4de2-885c-ad92f0aad022 req-44a2a355-88f7-4634-b3b9-131623d97134 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "name": "network-vif-plugged", "status": "completed", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:23.601135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-508070ba-924a-4de2-885c-ad92f0aad022 req-44a2a355-88f7-4634-b3b9-131623d97134 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:23.601706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-508070ba-924a-4de2-885c-ad92f0aad022 req-44a2a355-88f7-4634-b3b9-131623d97134 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:23.602240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-508070ba-924a-4de2-885c-ad92f0aad022 req-44a2a355-88f7-4634-b3b9-131623d97134 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:23.621861 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-508070ba-924a-4de2-885c-ad92f0aad022 req-44a2a355-88f7-4634-b3b9-131623d97134 service nova] Creating event network-vif-plugged:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 00:59:23.627780 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-508070ba-924a-4de2-885c-ad92f0aad022 req-44a2a355-88f7-4634-b3b9-131623d97134 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041178 Sep 11 00:59:23.628378 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 410/806] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:23.885624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d1388865-0e74-4903-9372-63a802f5ba0b tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01abde3a0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:23.912299 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-d1388865-0e74-4903-9372-63a802f5ba0b tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] HTTP exception thrown: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d1388865-0e74-4903-9372-63a802f5ba0b tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Request method failure captured: Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 HTTP/1.1 Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTLkje0ku8-NrmCLuPBxXvCX6qKVNf0-9bh2R03s3ReO1K_xl2ndNg3jD5goyeh1716r-v4iFup9qHHvTzH6TwcNRyvpxvgniNxfw0R5tpDTZjU45fgWfnVBsKeKWcfTJEMQx3yJlO0702CHTGr_KLk0XKijOrjYela9IpyploqkcFE Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:23.914572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 52cf4afdd2c741f8800360522a6d2f01 Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'ea760c3c-7264-44ab-b576-15af7dfb4731'} Sep 11 00:59:23.916136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. (HTTP 404) (Request-ID: req-5e54930f-4914-4e34-b8fc-b5d729729995) Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.917069 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:59:23.918437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 11 00:59:23.919853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d1388865-0e74-4903-9372-63a802f5ba0b tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Returning 404 to user: Snapshot ea760c3c-7264-44ab-b576-15af7dfb4731 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d1388865-0e74-4903-9372-63a802f5ba0b tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731" status: 404 len: 111 microversion: 2.1 time: 0.034607 Sep 11 00:59:23.921335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 397/807] 10.222.0.27 () {58 vars in 1251 bytes} [Sun Sep 11 00:59:23 2022] GET /compute/v2.1/os-snapshots/ea760c3c-7264-44ab-b576-15af7dfb4731 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:23.928570 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86331358-5c84-41a2-ae5b-8c0f51ecf74e tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01abc95e0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:23.998654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-86331358-5c84-41a2-ae5b-8c0f51ecf74e tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 202 len: 0 microversion: 2.1 time: 0.074667 Sep 11 00:59:23.999331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 411/808] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:59:23 2022] DELETE /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:59:24.008537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-cb546886-4c44-4941-8188-56bb01d20832 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01a92bb80>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:24.057574 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-cb546886-4c44-4941-8188-56bb01d20832 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 200 len: 344 microversion: 2.1 time: 0.053819 Sep 11 00:59:24.058091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 398/809] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:59:24 2022] GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 344 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:24.416889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:24.417613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:24.424217 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:24.424692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:24.425110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:24.425786 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:24.426231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:24.426644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:24.556981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:24.565047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:24.566492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:24.567683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b0d24551-1037-4560-8f1e-77f7e97ac6b4 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.155883 Sep 11 00:59:24.568237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 412/810] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:24 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:25.073306 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8cc3dda2-ead5-4e82-8b13-11d6be014b7d tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01abde160>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:25.101539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-8cc3dda2-ead5-4e82-8b13-11d6be014b7d tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] HTTP exception thrown: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8cc3dda2-ead5-4e82-8b13-11d6be014b7d tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Request method failure captured: Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 HTTP/1.1 Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTLkje0ku8-NrmCLuPBxXvCX6qKVNf0-9bh2R03s3ReO1K_xl2ndNg3jD5goyeh1716r-v4iFup9qHHvTzH6TwcNRyvpxvgniNxfw0R5tpDTZjU45fgWfnVBsKeKWcfTJEMQx3yJlO0702CHTGr_KLk0XKijOrjYela9IpyploqkcFE Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:25.103726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 52cf4afdd2c741f8800360522a6d2f01 Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8'} Sep 11 00:59:25.105264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. (HTTP 404) (Request-ID: req-0628a340-8929-4d36-8863-844def20297e) Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.106309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:59:25.108301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 11 00:59:25.109797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8cc3dda2-ead5-4e82-8b13-11d6be014b7d tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Returning 404 to user: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8cc3dda2-ead5-4e82-8b13-11d6be014b7d tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 404 len: 109 microversion: 2.1 time: 0.036249 Sep 11 00:59:25.111193 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 399/811] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:59:25 2022] GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:25.119879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-072db540-c63e-4a73-955d-f400b350c51f tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01abc95e0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:25.151963 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-072db540-c63e-4a73-955d-f400b350c51f tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] HTTP exception thrown: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-072db540-c63e-4a73-955d-f400b350c51f tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Request method failure captured: Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 HTTP/1.1 Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTLkje0ku8-NrmCLuPBxXvCX6qKVNf0-9bh2R03s3ReO1K_xl2ndNg3jD5goyeh1716r-v4iFup9qHHvTzH6TwcNRyvpxvgniNxfw0R5tpDTZjU45fgWfnVBsKeKWcfTJEMQx3yJlO0702CHTGr_KLk0XKijOrjYela9IpyploqkcFE Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:25.154056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 52cf4afdd2c741f8800360522a6d2f01 Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8'} Sep 11 00:59:25.155663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. (HTTP 404) (Request-ID: req-26a1b4df-d27c-46f2-972d-a0d957817f6d) Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.156859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 11 00:59:25.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 11 00:59:25.159655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-072db540-c63e-4a73-955d-f400b350c51f tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Returning 404 to user: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-072db540-c63e-4a73-955d-f400b350c51f tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 404 len: 109 microversion: 2.1 time: 0.039520 Sep 11 00:59:25.161001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 413/812] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 00:59:25 2022] DELETE /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:25.164407 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5ab499c2-1aa1-477f-bb57-43d75ec8a2c5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Calling method '.version_select at 0x7fc01a95af70>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:25.191432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-5ab499c2-1aa1-477f-bb57-43d75ec8a2c5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] HTTP exception thrown: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5ab499c2-1aa1-477f-bb57-43d75ec8a2c5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Request method failure captured: Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 HTTP/1.1 Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTLkje0ku8-NrmCLuPBxXvCX6qKVNf0-9bh2R03s3ReO1K_xl2ndNg3jD5goyeh1716r-v4iFup9qHHvTzH6TwcNRyvpxvgniNxfw0R5tpDTZjU45fgWfnVBsKeKWcfTJEMQx3yJlO0702CHTGr_KLk0XKijOrjYela9IpyploqkcFE Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:25.194081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_3de207e316eb493686e4d1b7592e94d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 3de207e316eb493686e4d1b7592e94d2 Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-326668502 Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 52cf4afdd2c741f8800360522a6d2f01 Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesSnapshotsTestJSON-326668502-project Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8'} Sep 11 00:59:25.195786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. (HTTP 404) (Request-ID: req-5d32b5db-9951-40d4-a186-fc804fcab400) Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.196670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 00:59:25.197852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 11 00:59:25.199051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5ab499c2-1aa1-477f-bb57-43d75ec8a2c5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] Returning 404 to user: Volume 9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5ab499c2-1aa1-477f-bb57-43d75ec8a2c5 tempest-VolumesSnapshotsTestJSON-326668502 tempest-VolumesSnapshotsTestJSON-326668502-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8" status: 404 len: 109 microversion: 2.1 time: 0.035190 Sep 11 00:59:25.200312 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 400/813] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 00:59:25 2022] GET /compute/v2.1/os-volumes/9ef7724a-9e4d-4f06-a4f2-fe55ac78a8a8 => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 00:59:25.596572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:25.597405 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:25.606443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:25.606961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:25.607402 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:25.608129 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:25.608584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:25.609009 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:25.810460 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:25.820489 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:25.821930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:25.823160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c5c2c8aa-742d-492e-ba4a-e9abcdfd319a tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1731 microversion: 2.20 time: 0.231657 Sep 11 00:59:25.823711 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 414/814] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:25 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1731 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:26.849721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:26.850238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:26.856358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:26.856829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:26.857265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:26.857954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:26.858404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:26.859185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:27.214968 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:27.223184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:27.225553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:27.226760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-77c7417a-b98b-4830-b5b5-cf1f3555cd52 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1718 microversion: 2.20 time: 0.382069 Sep 11 00:59:27.227333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 401/815] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 00:59:26 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1718 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 00:59:44.706912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-3b4e9852-2800-4c5e-8b64-773bb10d794c req-c8956825-d164-4a4a-9e29-46456d35ed8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:44.714683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:44.715247 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:44.717895 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:44.718688 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:44.725719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:44.726224 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:44.726702 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:44.729480 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=52,nova_api:DELETE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:44.730181 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=27 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:44.735442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-3b4e9852-2800-4c5e-8b64-773bb10d794c req-c8956825-d164-4a4a-9e29-46456d35ed8e service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:44.735893 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-3b4e9852-2800-4c5e-8b64-773bb10d794c req-c8956825-d164-4a4a-9e29-46456d35ed8e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:44.736826 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:44.737547 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:44.738072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:44.738908 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-3b4e9852-2800-4c5e-8b64-773bb10d794c req-c8956825-d164-4a4a-9e29-46456d35ed8e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:44.771534 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-3b4e9852-2800-4c5e-8b64-773bb10d794c req-c8956825-d164-4a4a-9e29-46456d35ed8e service nova] Creating event network-changed:549957cc-ef50-4e12-9608-80b215c4bfb4 for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c on hv-cinder-maste Sep 11 00:59:44.780810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-3b4e9852-2800-4c5e-8b64-773bb10d794c req-c8956825-d164-4a4a-9e29-46456d35ed8e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077355 Sep 11 00:59:44.782749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 402/816] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 00:59:44.787930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:59:44.886209 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-147258a6-54ef-4813-8427-d07a822cba29 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "DELETE /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 204 len: 0 microversion: 2.1 time: 0.185680 Sep 11 00:59:44.887371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 415/817] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:59:44 2022] DELETE /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 00:59:44.897889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:44.899175 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:44.908162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:44.909036 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:44.909847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:44.911144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:44.911940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:44.913132 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:45.062604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:45.071852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d968b71a-19ce-4348-aa6c-bb5fada357da tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.179590 Sep 11 00:59:45.072425 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 403/818] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:44 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:46.100264 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:46.101058 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:46.107896 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:46.108395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:46.108820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:46.109515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:46.109959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:46.110370 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:46.318119 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:46.326261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-af67879c-accf-4ccf-ab85-591ff5673b65 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1569 microversion: 2.1 time: 0.230604 Sep 11 00:59:46.327249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 416/819] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:46 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1569 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:47.326522 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-0ebbe69e-6bdd-42d6-87c4-0a4bdfcec6fb req-fd4bd324-98ae-48a8-a97a-415d06d48e8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77aec922-e7ba-4f8c-8931-b2ea825af43c", "name": "network-vif-deleted", "tag": "549957cc-ef50-4e12-9608-80b215c4bfb4"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:47.342045 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-0ebbe69e-6bdd-42d6-87c4-0a4bdfcec6fb req-fd4bd324-98ae-48a8-a97a-415d06d48e8a service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:47.342627 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-0ebbe69e-6bdd-42d6-87c4-0a4bdfcec6fb req-fd4bd324-98ae-48a8-a97a-415d06d48e8a service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:47.343873 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-0ebbe69e-6bdd-42d6-87c4-0a4bdfcec6fb req-fd4bd324-98ae-48a8-a97a-415d06d48e8a service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:47.360821 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-0ebbe69e-6bdd-42d6-87c4-0a4bdfcec6fb req-fd4bd324-98ae-48a8-a97a-415d06d48e8a service nova] Creating event network-vif-deleted:549957cc-ef50-4e12-9608-80b215c4bfb4 for instance 77aec922-e7ba-4f8c-8931-b2ea825af43c on hv-cinder-maste Sep 11 00:59:47.369815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-0ebbe69e-6bdd-42d6-87c4-0a4bdfcec6fb req-fd4bd324-98ae-48a8-a97a-415d06d48e8a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048812 Sep 11 00:59:47.370452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 404/820] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:47.615612 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:47.616821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:47.624561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:47.625391 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:47.626066 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:47.626912 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:47.627720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:47.628350 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:47.703775 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:47.713975 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7eb4ef59-d582-4306-8dfa-8dd27a8e3db6 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1382 microversion: 2.1 time: 0.367411 Sep 11 00:59:47.715069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 417/821] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:47 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1382 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:48.743076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:48.743904 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:48.750495 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:48.751115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:48.751618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:48.752313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:48.752762 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:48.753184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:48.790053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:48.790532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:48.791000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:48.791441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lazy-loading 'fault' on Instance uuid 77aec922-e7ba-4f8c-8931-b2ea825af43c {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:59:48.833448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '967e0ef4e6364804a58d0d59c22cd6a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd55d41063ec34fb6921c367025fa9af9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:48.842096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-732d7329-f8da-4880-b8de-ef61ce3a032c tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 200 len: 1387 microversion: 2.1 time: 0.104863 Sep 11 00:59:48.842337 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 405/822] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:48 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 1387 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:49.066260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:49.066990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:49.076440 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:49.077024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:49.077635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:49.079437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:49.080406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:49.080998 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:49.186575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lazy-loading 'services' on Instance uuid 28ecae2d-9fbd-42c1-932c-40406ca730b0 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 00:59:49.262400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-edb88118-1103-462d-a9ca-6fe0ec6cb935 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "DELETE /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0/os-volume_attachments/b8d2ce3d-f837-4c35-a7ad-3185ea06f91c" status: 202 len: 0 microversion: 2.1 time: 0.201318 Sep 11 00:59:49.263117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 418/823] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 00:59:49 2022] DELETE /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0/os-volume_attachments/b8d2ce3d-f837-4c35-a7ad-3185ea06f91c => generated 0 bytes in 203 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:59:49.866188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:49.866865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] [instance: 77aec922-e7ba-4f8c-8931-b2ea825af43c] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:49.873809 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:49.874376 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:49.874853 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:49.875556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:49.876403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:49.880019 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:49.896719 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] HTTP exception thrown: Instance 77aec922-e7ba-4f8c-8931-b2ea825af43c could not be found. Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Request method failure captured: Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c HTTP/1.1 Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTHPyhIB2O2RbkNRI65XqA8QFmO_OZw5KYyRRMQchDQJiprLvb4VP9-oY17xL4Q0juS5BG2UnKZnTdPVlF0CRA4sNVoweM7q9qkGq6jpDZFyU_VtJkWFvj-Jqs4A_Kz6PGdm4l6wa-aE79qp5H3ZX9hkjLNlKmF9fCkJXgcJbrhn7rE Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: d55d41063ec34fb6921c367025fa9af9 Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-TestShelveInstance-1577537656 Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 00:59:49.898459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d55d41063ec34fb6921c367025fa9af9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d55d41063ec34fb6921c367025fa9af9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d55d41063ec34fb6921c367025fa9af9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d55d41063ec34fb6921c367025fa9af9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-TestShelveInstance-1577537656 Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: d55d41063ec34fb6921c367025fa9af9 Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-TestShelveInstance-1577537656 Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-TestShelveInstance-1577537656-project Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 967e0ef4e6364804a58d0d59c22cd6a8 Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-TestShelveInstance-1577537656-project Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 77aec922-e7ba-4f8c-8931-b2ea825af43c could not be found. Sep 11 00:59:49.900369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '77aec922-e7ba-4f8c-8931-b2ea825af43c'} Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77aec922-e7ba-4f8c-8931-b2ea825af43c could not be found. Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:49.901217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77aec922-e7ba-4f8c-8931-b2ea825af43c could not be found. Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77aec922-e7ba-4f8c-8931-b2ea825af43c could not be found. Sep 11 00:59:49.902518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 00:59:49.903705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Returning 404 to user: Instance 77aec922-e7ba-4f8c-8931-b2ea825af43c could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:49.903705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-03a75f09-d0ff-4e13-8e72-e76c0fbb191d tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c" status: 404 len: 111 microversion: 2.1 time: 0.038744 Sep 11 00:59:49.903705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 406/824] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:49 2022] GET /compute/v2.1/servers/77aec922-e7ba-4f8c-8931-b2ea825af43c => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 00:59:50.085341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fc0a4044-1883-4e3c-a435-b57c80ea0cb3 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] Calling method '.version_select at 0x7fc01aa6e0d0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:50.112192 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fc0a4044-1883-4e3c-a435-b57c80ea0cb3 tempest-TestShelveInstance-1577537656 tempest-TestShelveInstance-1577537656-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1739406180" status: 202 len: 0 microversion: 2.1 time: 0.034462 Sep 11 00:59:50.113125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 419/825] 10.222.0.27 () {58 vars in 1254 bytes} [Sun Sep 11 00:59:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1739406180 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 00:59:50.409290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:50.410263 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:50.416799 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:50.417511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:50.418118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:50.419034 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:50.419655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:50.420245 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:50.459122 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 00:59:50.541277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-e944eaf9-109d-43c3-986f-4a0d934bcb47 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "DELETE /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 204 len: 0 microversion: 2.1 time: 0.136299 Sep 11 00:59:50.542330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 407/826] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 00:59:50 2022] DELETE /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 00:59:50.551340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:50.552113 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:50.559588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:50.560076 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:50.560506 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:50.561197 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:50.561824 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:50.562240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:50.713382 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:50.722306 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-78c9399d-287c-45d3-9a33-aa024a98daa7 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1514 microversion: 2.1 time: 0.175365 Sep 11 00:59:50.723387 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 420/827] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:50 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1514 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:51.749208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:51.749763 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:51.756769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:51.757437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:51.758113 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:51.759249 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:51.759810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:51.760501 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:51.912121 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:51.922545 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-00173982-5ac7-4a0d-b9db-9e6965bd959c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1514 microversion: 2.1 time: 0.177665 Sep 11 00:59:51.923681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 408/828] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:51 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1514 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:52.949053 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:52.949884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:52.961203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:52.962743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:52.963490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:52.964413 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:52.965048 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:52.965661 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:53.103102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 00:59:53.115397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-47a54881-9246-443c-8bde-83ee9b8e1954 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 200 len: 1472 microversion: 2.1 time: 0.170589 Sep 11 00:59:53.116138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 421/829] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:52 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 1472 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 00:59:53.214058 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-71ba6bd9-4136-4ea6-8fa9-602e15c24857 req-ea280ff2-361f-4adf-bc2e-bb34dec48bbe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "28ecae2d-9fbd-42c1-932c-40406ca730b0", "name": "network-vif-deleted", "tag": "1717a3c3-263a-47c4-85ce-30d613700835"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 00:59:53.228796 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-71ba6bd9-4136-4ea6-8fa9-602e15c24857 req-ea280ff2-361f-4adf-bc2e-bb34dec48bbe service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:53.230120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-71ba6bd9-4136-4ea6-8fa9-602e15c24857 req-ea280ff2-361f-4adf-bc2e-bb34dec48bbe service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:53.231735 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-71ba6bd9-4136-4ea6-8fa9-602e15c24857 req-ea280ff2-361f-4adf-bc2e-bb34dec48bbe service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:53.247330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-71ba6bd9-4136-4ea6-8fa9-602e15c24857 req-ea280ff2-361f-4adf-bc2e-bb34dec48bbe service nova] Creating event network-vif-deleted:1717a3c3-263a-47c4-85ce-30d613700835 for instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 on hv-cinder-maste Sep 11 00:59:53.252607 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-71ba6bd9-4136-4ea6-8fa9-602e15c24857 req-ea280ff2-361f-4adf-bc2e-bb34dec48bbe service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042679 Sep 11 00:59:53.253576 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 409/830] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 00:59:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 00:59:54.140838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 00:59:54.141568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: 28ecae2d-9fbd-42c1-932c-40406ca730b0] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 00:59:54.148278 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:54.148768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:54.149232 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:54.149905 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 00:59:54.150533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 00:59:54.151155 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 00:59:54.166087 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] HTTP exception thrown: Instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 could not be found. Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Request method failure captured: Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 HTTP/1.1 Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTLGzgZSxnOVzz73ILtWuwtVSaWoyJCnZczekl19KlRKj086-5vrDH_l0s-CUb7Sij7KnQTOct-wZvPnQDM7z7C-_llUZYxTBaLzxdGzr1tHKHOl_oC5CVrE2nS6LK36H8F8N3pW7cV0B6fyQyb-zuDTWSddkeGBRKnrrqrsY9wWiJI Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: de36b740b8234aca8317d5e4440decec Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesSnapshotTestJSON-649966533 Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 00:59:54.167804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/de36b740b8234aca8317d5e4440decec"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/de36b740b8234aca8317d5e4440decec"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/de36b740b8234aca8317d5e4440decec"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_de36b740b8234aca8317d5e4440decec"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesSnapshotTestJSON-649966533 Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: de36b740b8234aca8317d5e4440decec Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-649966533 Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesSnapshotTestJSON-649966533-project Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 78a2199d3f1d4bc5b1f988ae06c0d609 Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesSnapshotTestJSON-649966533-project Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 could not be found. Sep 11 00:59:54.169414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '28ecae2d-9fbd-42c1-932c-40406ca730b0'} Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 could not be found. Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 00:59:54.170335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 could not be found. Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 could not be found. Sep 11 00:59:54.171876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 00:59:54.173240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Returning 404 to user: Instance 28ecae2d-9fbd-42c1-932c-40406ca730b0 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 00:59:54.173240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f5a2153b-1b07-411f-90c5-316444785107 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0" status: 404 len: 111 microversion: 2.1 time: 0.032299 Sep 11 00:59:54.173240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 422/831] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 00:59:54 2022] GET /compute/v2.1/servers/28ecae2d-9fbd-42c1-932c-40406ca730b0 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 11 01:00:02.110137 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1561349688", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "networks": [{"uuid": "e7aac99f-395b-41fd-b96e-56bde33609b0"}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:02.170345 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] validate_networks() for [('e7aac99f-395b-41fd-b96e-56bde33609b0', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:00:02.756856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:00:02.757277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:00:02.757747 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:00:02.758153 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:00:02.758562 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:00:02.767226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:02.767708 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:02.768134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:02.768847 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:02.769987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:02.770579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:02.787129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Getting quotas for project de36b740b8234aca8317d5e4440decec. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:00:02.791101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Getting quotas for user 78a2199d3f1d4bc5b1f988ae06c0d609 and project de36b740b8234aca8317d5e4440decec. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:00:02.804032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:00:02.806427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:00:02.852765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b4c3bc89-bc3f-47b3-8560-2e568972db9a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.749576 Sep 11 01:00:02.853401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 410/832] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:00:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 751 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:00:02.865265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-02f2625e-5f33-484b-a65b-cfdddbfe779a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:02.866212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-02f2625e-5f33-484b-a65b-cfdddbfe779a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:02.920611 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-02f2625e-5f33-484b-a65b-cfdddbfe779a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:02.928407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-02f2625e-5f33-484b-a65b-cfdddbfe779a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1219 microversion: 2.1 time: 0.067512 Sep 11 01:00:02.928969 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 423/833] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:02 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1219 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:00:03.953060 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:03.953680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:03.960876 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:03.961417 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:03.962062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:03.962963 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:03.963421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:03.963850 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:04.042108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:04.051730 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a0cd8afa-4c8d-4be9-93bc-084886819bc0 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1279 microversion: 2.1 time: 0.102266 Sep 11 01:00:04.052362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 411/834] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:03 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1279 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:05.075335 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:05.075949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:05.078512 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=49,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:05.085163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:05.085906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:05.086523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:05.087265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:05.087759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:05.088190 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:05.324727 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:05.332822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a0f0a758-4f59-42f0-abf1-03e02dc85c34 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.261581 Sep 11 01:00:05.333613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 424/835] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:05 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:00:05.462656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-f1c574d9-e1ae-404b-b5dd-53bca7ebc525 req-607f32be-fbca-451a-a1f5-5db5fc4bf90d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbd5152f-57bc-4ad1-8009-70681c4097d7", "tag": "581e085b-9fa5-4985-97c1-4dfd99d9a05d"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:05.475504 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f1c574d9-e1ae-404b-b5dd-53bca7ebc525 req-607f32be-fbca-451a-a1f5-5db5fc4bf90d service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:05.476462 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f1c574d9-e1ae-404b-b5dd-53bca7ebc525 req-607f32be-fbca-451a-a1f5-5db5fc4bf90d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:05.477179 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-f1c574d9-e1ae-404b-b5dd-53bca7ebc525 req-607f32be-fbca-451a-a1f5-5db5fc4bf90d service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:05.489775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-f1c574d9-e1ae-404b-b5dd-53bca7ebc525 req-607f32be-fbca-451a-a1f5-5db5fc4bf90d service nova] Creating event network-changed:581e085b-9fa5-4985-97c1-4dfd99d9a05d for instance fbd5152f-57bc-4ad1-8009-70681c4097d7 on hv-cinder-maste Sep 11 01:00:05.494893 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-f1c574d9-e1ae-404b-b5dd-53bca7ebc525 req-607f32be-fbca-451a-a1f5-5db5fc4bf90d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037562 Sep 11 01:00:05.495473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 412/836] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:00:06.355956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:06.356565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:06.362869 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:06.363660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:06.364097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:06.364804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:06.365265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:06.365669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:06.493412 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:06.500964 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5fabdfbc-459a-470c-970d-00fe29fcc0ca tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.149022 Sep 11 01:00:06.501531 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 425/837] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:06 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:07.526460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:07.527466 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:07.534170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:07.535416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:07.536280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:07.537322 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:07.537535 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:07.537994 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:07.679786 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:07.687279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-50e514f6-51fa-4435-bafa-67307f93d764 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.165958 Sep 11 01:00:07.687837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 413/838] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:07 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:08.710980 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:08.711585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:08.717477 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:08.717954 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:08.718379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:08.719081 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:08.719526 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:08.719941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:08.849682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:08.857392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b5d31605-a8a8-4c4b-bf92-3c458af00492 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.150980 Sep 11 01:00:08.857956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 426/839] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:08 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:09.881729 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:09.882267 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:09.888565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:09.889051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:09.889487 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:09.890184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:09.890638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:09.891090 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:10.027097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:10.035153 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c56892db-cf79-4f57-8752-62a6ef55ce4f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.157838 Sep 11 01:00:10.035728 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 414/840] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:09 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:11.057320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:11.057941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:11.064280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:11.064819 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:11.065253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:11.066671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:11.067158 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:11.067584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:11.199756 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:11.207623 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c39d63b6-1cf5-420e-a8c9-4705d9dc0a99 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.154184 Sep 11 01:00:11.208194 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 427/841] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:11 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:12.232195 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:12.232947 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:12.240529 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.241244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.241775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.242485 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.244694 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.245341 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.377082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:12.385734 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-22033ed7-d759-48fe-ac46-5d45d779ffe5 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.158804 Sep 11 01:00:12.386307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 415/842] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:12 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:12.677850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:12.680075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:12.686416 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.686930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.687366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.688050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.688500 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.688921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.717625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1ce88057-4b09-4824-a201-851b2d0df843 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.044118 Sep 11 01:00:12.718325 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 428/843] 10.222.0.27 () {60 vars in 1343 bytes} [Sun Sep 11 01:00:12 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:12.785776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:12.786361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:12.788348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:12.793736 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.794232 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.794674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.795400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.795866 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.796666 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.799406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:12.823991 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:00:12.891785 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-d5af7e6a-af6d-4a2f-bf88-fd10e6f85915 req-69b1d6b5-bf64-43bf-9e8a-62b8e7b4570b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbd5152f-57bc-4ad1-8009-70681c4097d7", "name": "network-vif-plugged", "status": "completed", "tag": "581e085b-9fa5-4985-97c1-4dfd99d9a05d"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:12.902407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d5af7e6a-af6d-4a2f-bf88-fd10e6f85915 req-69b1d6b5-bf64-43bf-9e8a-62b8e7b4570b service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.903154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d5af7e6a-af6d-4a2f-bf88-fd10e6f85915 req-69b1d6b5-bf64-43bf-9e8a-62b8e7b4570b service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.903911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-d5af7e6a-af6d-4a2f-bf88-fd10e6f85915 req-69b1d6b5-bf64-43bf-9e8a-62b8e7b4570b service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.910705 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-df50522c-23ef-4420-9208-2978bf065d1e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 204 len: 0 microversion: 2.20 time: 0.129610 Sep 11 01:00:12.915447 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 416/844] 10.222.0.27 () {60 vars in 1280 bytes} [Sun Sep 11 01:00:12 2022] DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 11 01:00:12.920394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-d5af7e6a-af6d-4a2f-bf88-fd10e6f85915 req-69b1d6b5-bf64-43bf-9e8a-62b8e7b4570b service nova] Creating event network-vif-plugged:581e085b-9fa5-4985-97c1-4dfd99d9a05d for instance fbd5152f-57bc-4ad1-8009-70681c4097d7 on hv-cinder-maste Sep 11 01:00:12.921116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:12.921830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:12.926038 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-d5af7e6a-af6d-4a2f-bf88-fd10e6f85915 req-69b1d6b5-bf64-43bf-9e8a-62b8e7b4570b service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037159 Sep 11 01:00:12.926891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 429/845] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:00:12.930824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.931330 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.931774 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:12.932508 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:12.932953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:12.933381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:13.067072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:13.075767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:13.077201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:13.078391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fcbe6f5f-32d0-49c0-9675-ed82bb2dda9e tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.161794 Sep 11 01:00:13.078971 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 417/846] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:12 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 01:00:13.406446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:13.411226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:13.411933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:13.418393 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:13.418898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:13.419336 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:13.420049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:13.420504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:13.420924 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:13.551904 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:13.559576 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2b10ff3b-0cc3-425c-9bb2-df0641a37250 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.154325 Sep 11 01:00:13.560137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 430/847] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:13 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:14.105043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:14.105680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:14.113690 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:14.114182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:14.114619 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:14.115347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:14.115806 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:14.116234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:14.252458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:14.258045 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:14.263843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:14.265300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:14.266583 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9293040f-8f09-421f-8bf6-26cf919c6f73 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.166086 Sep 11 01:00:14.267180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 418/848] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:14 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 01:00:14.582338 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:14.582990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:14.589286 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:14.589778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:14.590270 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:14.591002 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:14.591815 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:14.591966 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:14.722372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:14.729990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1192899c-91e5-47a9-9b21-9427d4bba58c tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1319 microversion: 2.1 time: 0.152513 Sep 11 01:00:14.730559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 431/849] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:14 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:15.292701 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:15.293317 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:15.295855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:15.301996 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:15.302505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:15.302951 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:15.303654 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:15.304109 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:15.304537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:15.561096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:15.570226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:15.571706 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:15.572875 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d1cb3567-b745-4f34-95c2-870bafe13e64 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1724 microversion: 2.20 time: 0.284162 Sep 11 01:00:15.573428 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 419/850] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:15 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1724 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 01:00:15.754949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:15.755792 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:15.758809 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:15.768825 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:15.770191 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:15.770698 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:15.772104 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:15.772686 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:15.773173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.294271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-0580e6af-bdf9-422c-b50f-9cb24ff91682 req-101a0d00-1469-4146-8c87-c1dd7adfe196 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54f4bbc1-5800-4bc9-9ea6-c9a3cd862347", "name": "network-vif-deleted", "tag": "5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:16.295071 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:16.306437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9079ace7-9b20-4fd1-9fe0-62119a911e06 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1507 microversion: 2.1 time: 0.555480 Sep 11 01:00:16.307084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 432/851] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:15 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1507 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:16.311761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-0580e6af-bdf9-422c-b50f-9cb24ff91682 req-101a0d00-1469-4146-8c87-c1dd7adfe196 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:16.312384 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-0580e6af-bdf9-422c-b50f-9cb24ff91682 req-101a0d00-1469-4146-8c87-c1dd7adfe196 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:16.312841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-0580e6af-bdf9-422c-b50f-9cb24ff91682 req-101a0d00-1469-4146-8c87-c1dd7adfe196 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.332524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8649ad14-f3b7-40ca-b3ec-ade6810f9e86", "device": "/dev/sdb"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:16.335988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:16.342555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:16.343358 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-0580e6af-bdf9-422c-b50f-9cb24ff91682 req-101a0d00-1469-4146-8c87-c1dd7adfe196 service nova] Creating event network-vif-deleted:5d0b683e-8ce0-4d8f-a5a3-6d54b3c2bd9a for instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 on hv-cinder-maste Sep 11 01:00:16.343513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:16.344160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.345156 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:16.345719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:16.346237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.354047 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-0580e6af-bdf9-422c-b50f-9cb24ff91682 req-101a0d00-1469-4146-8c87-c1dd7adfe196 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064285 Sep 11 01:00:16.355226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 420/852] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:00:16.604552 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:16.605385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:16.612059 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:16.613512 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:16.614318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.615216 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:16.615818 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:16.616399 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.656213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:16.656867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:16.657116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:16.657993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lazy-loading 'fault' on Instance uuid 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:00:16.676296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e22fff0b-ca18-40bb-8c0a-5dbab6b87eed tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "POST /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.349269 Sep 11 01:00:16.677108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 433/853] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 01:00:16 2022] POST /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7/os-volume_attachments => generated 194 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:00:16.704307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:16.712855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:16.714405 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '73a3d3c48913474ab8fa8fedd1bc7d03', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd9221b2e81a46f69864e02ec4d42672', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:16.715602 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-239872f3-9674-44df-9de9-28a4a667aa80 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 200 len: 1372 microversion: 2.20 time: 0.119386 Sep 11 01:00:16.719064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 421/854] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:16 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 1372 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 11 01:00:17.743972 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:17.744700 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:17.751437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:17.751921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:17.752357 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:17.754073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:17.754529 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:17.754982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:17.769386 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] HTTP exception thrown: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Request method failure captured: Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 HTTP/1.1 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFh7LqKP94hM4-KTW8o175jD3zotXAsDceOYAGcfUA4X4HFr7u0MTuULetTgdHMY9LAcSb8D8pljyJ-YTXo3ITsO2j7rVAonZoyAn6wNKepm-7hai-7Hfpk9sfmJyx8xRg1ZHt0EPAu5pez6rZ4rAig9k3SUOLp1byI1JCSrGdbVH8 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Openstack-Nova-Api-Version: 2.20 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:00:17.771178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 73a3d3c48913474ab8fa8fedd1bc7d03 Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '54f4bbc1-5800-4bc9-9ea6-c9a3cd862347'} Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:00:17.777548 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:17.779299 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:17.780731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Returning 404 to user: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:00:17.780731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4902d40f-b8dc-4e4f-abdf-2b9b110f806b tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 404 len: 111 microversion: 2.20 time: 0.032933 Sep 11 01:00:17.780731 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 434/855] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:17 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 11 01:00:17.781738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f576d8d5-260f-46b4-8da3-0e66b35068ff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '.version_select at 0x7fc01ac66af0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:17.809553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f576d8d5-260f-46b4-8da3-0e66b35068ff tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1815041047" status: 204 len: 0 microversion: 2.20 time: 0.031791 Sep 11 01:00:17.810103 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 422/856] 10.222.0.27 () {60 vars in 1262 bytes} [Sun Sep 11 01:00:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1815041047 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 11 01:00:19.586517 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:19.587094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:19.593411 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:19.593891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:19.594321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:19.595034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:19.595497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:19.595921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:19.609755 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] HTTP exception thrown: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Request method failure captured: Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 HTTP/1.1 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFh7LqKP94hM4-KTW8o175jD3zotXAsDceOYAGcfUA4X4HFr7u0MTuULetTgdHMY9LAcSb8D8pljyJ-YTXo3ITsO2j7rVAonZoyAn6wNKepm-7hai-7Hfpk9sfmJyx8xRg1ZHt0EPAu5pez6rZ4rAig9k3SUOLp1byI1JCSrGdbVH8 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Openstack-Nova-Api-Version: 2.20 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:00:19.611328 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 73a3d3c48913474ab8fa8fedd1bc7d03 Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.613283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '54f4bbc1-5800-4bc9-9ea6-c9a3cd862347'} Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:00:19.614167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.616641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.618769 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Returning 404 to user: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:00:19.618769 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fdd557b8-e7fe-4702-a997-d5fb2d463125 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 404 len: 111 microversion: 2.20 time: 0.030186 Sep 11 01:00:19.618769 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 435/857] 10.222.0.27 () {60 vars in 1280 bytes} [Sun Sep 11 01:00:19 2022] DELETE /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 11 01:00:19.622169 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:19.624695 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:19.630200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:19.630687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:19.631152 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:19.631855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:19.632308 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:19.632733 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:19.646290 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] HTTP exception thrown: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Request method failure captured: Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 HTTP/1.1 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFh7LqKP94hM4-KTW8o175jD3zotXAsDceOYAGcfUA4X4HFr7u0MTuULetTgdHMY9LAcSb8D8pljyJ-YTXo3ITsO2j7rVAonZoyAn6wNKepm-7hai-7Hfpk9sfmJyx8xRg1ZHt0EPAu5pez6rZ4rAig9k3SUOLp1byI1JCSrGdbVH8 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Openstack-Nova-Api-Version: 2.20 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:00:19.647960 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 73a3d3c48913474ab8fa8fedd1bc7d03 Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.649624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '54f4bbc1-5800-4bc9-9ea6-c9a3cd862347'} Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:00:19.650539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. Sep 11 01:00:19.652061 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:19.653472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Returning 404 to user: Instance 54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:00:19.653472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f1582a9c-afe0-4293-bbeb-51399c5be770 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347" status: 404 len: 111 microversion: 2.20 time: 0.031753 Sep 11 01:00:19.653472 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 423/858] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:19 2022] GET /compute/v2.1/servers/54f4bbc1-5800-4bc9-9ea6-c9a3cd862347 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 11 01:00:20.798705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:20.799213 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:20.806845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:20.807326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:20.807746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:20.808449 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:20.808892 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:20.809333 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:20.823226 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] HTTP exception thrown: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Request method failure captured: Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 HTTP/1.1 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTFh7LqKP94hM4-KTW8o175jD3zotXAsDceOYAGcfUA4X4HFr7u0MTuULetTgdHMY9LAcSb8D8pljyJ-YTXo3ITsO2j7rVAonZoyAn6wNKepm-7hai-7Hfpk9sfmJyx8xRg1ZHt0EPAu5pez6rZ4rAig9k3SUOLp1byI1JCSrGdbVH8 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Openstack-Nova-Api-Version: 2.20 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:00:20.824986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 73a3d3c48913474ab8fa8fedd1bc7d03 Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.826726 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '7ed280f0-7beb-47b6-a543-0e3eb7083067'} Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:00:20.827773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.829295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.831875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Returning 404 to user: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:00:20.831875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a7fdcead-126d-49ef-8a6c-b29f0e2901a3 tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 404 len: 111 microversion: 2.20 time: 0.031551 Sep 11 01:00:20.831875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 436/859] 10.222.0.27 () {60 vars in 1280 bytes} [Sun Sep 11 01:00:20 2022] DELETE /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 11 01:00:20.835877 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:20.836486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] [instance: 7ed280f0-7beb-47b6-a543-0e3eb7083067] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:20.842856 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:20.843334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:20.843764 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:20.844441 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:20.844890 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:20.845307 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:20.859632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] HTTP exception thrown: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Request method failure captured: Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 HTTP/1.1 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTFh7LqKP94hM4-KTW8o175jD3zotXAsDceOYAGcfUA4X4HFr7u0MTuULetTgdHMY9LAcSb8D8pljyJ-YTXo3ITsO2j7rVAonZoyAn6wNKepm-7hai-7Hfpk9sfmJyx8xRg1ZHt0EPAu5pez6rZ4rAig9k3SUOLp1byI1JCSrGdbVH8 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Openstack-Nova-Api-Version: 2.20 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:00:20.861247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fd9221b2e81a46f69864e02ec4d42672"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: fd9221b2e81a46f69864e02ec4d42672 Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-728548594 Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 73a3d3c48913474ab8fa8fedd1bc7d03 Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeShelveTestJSON-728548594-project Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.863067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '7ed280f0-7beb-47b6-a543-0e3eb7083067'} Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:00:20.864813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. Sep 11 01:00:20.866460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:20.868089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] Returning 404 to user: Instance 7ed280f0-7beb-47b6-a543-0e3eb7083067 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:00:20.868089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-68bb1179-21bf-4922-8fd1-473b73b641ae tempest-AttachVolumeShelveTestJSON-728548594 tempest-AttachVolumeShelveTestJSON-728548594-project] 10.222.0.27 "GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067" status: 404 len: 111 microversion: 2.20 time: 0.031013 Sep 11 01:00:20.868089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 424/860] 10.222.0.27 () {60 vars in 1277 bytes} [Sun Sep 11 01:00:20 2022] GET /compute/v2.1/servers/7ed280f0-7beb-47b6-a543-0e3eb7083067 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 11 01:00:29.700906 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-89fd80ed-04a6-4dcf-b460-9a956076cbfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: .version_select at 0x7fc01aa96040>, body: {"keypair": {"name": "tempest-keypair-993355735", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMShiBNG8LEbhaoHUHRxfnQ9wRVoppNXJFbPUE6dZaYVEr5dtfiliO4pC/eyHJ+rORGecgHO990vdv08JdyIGdbKrIV1sZeinNNwHDBKbkHu/MiER4dIuo4r5vXGDhBkmQ=="}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:29.717635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-89fd80ed-04a6-4dcf-b460-9a956076cbfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for project 15810c36aaa44c5fbc87197f873a6260. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:00:29.721796 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-89fd80ed-04a6-4dcf-b460-9a956076cbfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for user 58b3528fd06b4d438caa92fd88383ed5 and project 15810c36aaa44c5fbc87197f873a6260. Resources: {'key_pairs'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:00:29.745316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-89fd80ed-04a6-4dcf-b460-9a956076cbfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.167866 Sep 11 01:00:29.745921 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 437/861] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 01:00:29 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:00:31.099606 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1466965751", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "049b6c8b-1072-47b9-b740-d7f0f662cb7f"}], "security_groups": [{"name": "tempest-securitygroup--1133749187"}], "key_name": "tempest-keypair-993355735", "user_data": "}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:31.109340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=14,nova_api:DELETE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:31.109993 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=20 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:31.212333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] validate_networks() for [('049b6c8b-1072-47b9-b740-d7f0f662cb7f', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:00:31.586010 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:00:31.586434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:00:31.586938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:00:31.587348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:00:31.587761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:00:31.596473 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:31.596955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:31.597950 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:31.598846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:31.599309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:31.599736 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:31.616985 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for project 15810c36aaa44c5fbc87197f873a6260. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:00:31.620918 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for user 58b3528fd06b4d438caa92fd88383ed5 and project 15810c36aaa44c5fbc87197f873a6260. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:00:31.632108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:00:31.634401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:00:31.688864 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-90e80387-c19a-4344-b97d-06bfecb61b02 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.594749 Sep 11 01:00:31.689703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 425/862] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:00:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 596 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:00:31.701077 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fb725e2b-50af-45b7-ab09-05f81ace85f4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:31.701691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fb725e2b-50af-45b7-ab09-05f81ace85f4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:31.704345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:31.771240 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fb725e2b-50af-45b7-ab09-05f81ace85f4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:31.779080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fb725e2b-50af-45b7-ab09-05f81ace85f4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1237 microversion: 2.1 time: 0.081789 Sep 11 01:00:31.779763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 438/863] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:31 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1237 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:32.802469 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:32.803126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:32.809938 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:32.810427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:32.810896 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:32.811873 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:32.812329 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:32.812752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:32.881813 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:32.889710 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3f9c5f51-caa9-48fc-af66-9ff0b6e7909c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1241 microversion: 2.1 time: 0.092065 Sep 11 01:00:32.890293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 426/864] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:32 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1241 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:33.911958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:33.912585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:33.919170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:33.919660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:33.920098 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:33.920802 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:33.921257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:33.921697 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:33.982257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:33.990475 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9f952b79-b860-4e45-b5b8-c7dc1433df88 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1241 microversion: 2.1 time: 0.082604 Sep 11 01:00:33.991073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 439/865] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:33 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1241 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:35.013502 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:35.014112 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:35.021917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:35.021917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:35.021917 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:35.022354 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:35.022824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:35.023266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:35.087237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:35.095449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d076fc5f-3954-4786-b4f2-ff86c1499723 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1289 microversion: 2.1 time: 0.085767 Sep 11 01:00:35.096012 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 427/866] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:35 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1289 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:36.123065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:36.123705 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:36.133471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:36.133969 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:36.134405 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:36.135262 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:36.135734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:36.136163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:36.454626 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:36.464082 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-325d3d1c-ed07-4e11-b536-e849307a70e9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1363 microversion: 2.1 time: 0.344769 Sep 11 01:00:36.465717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 440/867] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:36 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1363 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:36.837215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-52083a6d-ec1d-45e2-80b2-0d9362e094b5 req-5272a8e6-cb12-493f-b293-d8532954ce99 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0875c4c-2396-4199-bf90-c505cd4ab7f8", "tag": "92f73122-c3f2-4b96-9d29-f12911010f02"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:36.854305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-52083a6d-ec1d-45e2-80b2-0d9362e094b5 req-5272a8e6-cb12-493f-b293-d8532954ce99 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:36.856126 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-52083a6d-ec1d-45e2-80b2-0d9362e094b5 req-5272a8e6-cb12-493f-b293-d8532954ce99 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:36.860849 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-52083a6d-ec1d-45e2-80b2-0d9362e094b5 req-5272a8e6-cb12-493f-b293-d8532954ce99 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:36.891008 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-52083a6d-ec1d-45e2-80b2-0d9362e094b5 req-5272a8e6-cb12-493f-b293-d8532954ce99 service nova] Creating event network-changed:92f73122-c3f2-4b96-9d29-f12911010f02 for instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 on hv-cinder-maste Sep 11 01:00:36.897449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-52083a6d-ec1d-45e2-80b2-0d9362e094b5 req-5272a8e6-cb12-493f-b293-d8532954ce99 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064855 Sep 11 01:00:36.898496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 428/868] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:00:37.488729 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:37.489347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:37.495671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:37.496146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:37.496610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:37.497295 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:37.497738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:37.498153 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:37.625229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:37.633471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fe9a9ecb-d1c3-4ad3-a43b-e99f2f33c814 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1363 microversion: 2.1 time: 0.149131 Sep 11 01:00:37.634024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 441/869] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:37 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:38.297070 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:38.297826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:38.306375 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:38.307033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:38.307565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:38.308445 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:38.308901 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:38.309375 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:38.398209 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lazy-loading 'services' on Instance uuid fbd5152f-57bc-4ad1-8009-70681c4097d7 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:00:38.474437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c3a1552a-a172-4f45-af89-881abd91487f tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7/os-volume_attachments/8649ad14-f3b7-40ca-b3ec-ade6810f9e86" status: 202 len: 0 microversion: 2.1 time: 0.181184 Sep 11 01:00:38.475333 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 429/870] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 01:00:38 2022] DELETE /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7/os-volume_attachments/8649ad14-f3b7-40ca-b3ec-ade6810f9e86 => generated 0 bytes in 184 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:00:38.656951 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:38.657491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:38.664511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:38.665004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:38.665440 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:38.666134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:38.666584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:38.667041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:38.800458 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:38.808552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9c4f7bca-15a8-425d-a0d8-8ca7b5a3ec71 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1363 microversion: 2.1 time: 0.155748 Sep 11 01:00:38.809117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 442/871] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:38 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:39.604872 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:39.605459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:39.611604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:39.612093 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:39.612537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:39.613223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:39.613680 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:39.614103 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:39.639454 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:00:39.715040 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-de0f1c68-9bcf-4b7f-8b7e-c2e0df1c623a tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 204 len: 0 microversion: 2.1 time: 0.114553 Sep 11 01:00:39.715620 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 430/872] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:00:39 2022] DELETE /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 01:00:39.725684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:39.726305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:39.733175 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:39.733648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:39.734074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:39.734754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:39.735232 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:39.735644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:39.833416 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:39.834144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:39.840013 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:39.840498 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:39.840931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:39.841624 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:39.842075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:39.842498 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:39.863684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:39.872157 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e7cb5499-c287-4fef-b9fa-ad1d0f49e693 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1513 microversion: 2.1 time: 0.152081 Sep 11 01:00:39.872724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 443/873] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:39 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1513 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:39.974995 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:39.982518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5a076ba6-6323-4866-8f2c-5f3145da42d3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1363 microversion: 2.1 time: 0.153407 Sep 11 01:00:39.983118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 431/874] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:39 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1363 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:00:40.557991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-086c939c-2f66-4782-8231-1b1fa1f24b2b req-f3b98ced-dedd-4b7b-b586-b8de84e9c2e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0875c4c-2396-4199-bf90-c505cd4ab7f8", "name": "network-vif-plugged", "status": "completed", "tag": "92f73122-c3f2-4b96-9d29-f12911010f02"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:40.568932 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-086c939c-2f66-4782-8231-1b1fa1f24b2b req-f3b98ced-dedd-4b7b-b586-b8de84e9c2e9 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:40.569451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-086c939c-2f66-4782-8231-1b1fa1f24b2b req-f3b98ced-dedd-4b7b-b586-b8de84e9c2e9 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:40.570061 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-086c939c-2f66-4782-8231-1b1fa1f24b2b req-f3b98ced-dedd-4b7b-b586-b8de84e9c2e9 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:40.583746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-086c939c-2f66-4782-8231-1b1fa1f24b2b req-f3b98ced-dedd-4b7b-b586-b8de84e9c2e9 service nova] Creating event network-vif-plugged:92f73122-c3f2-4b96-9d29-f12911010f02 for instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 on hv-cinder-maste Sep 11 01:00:40.590397 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-086c939c-2f66-4782-8231-1b1fa1f24b2b req-f3b98ced-dedd-4b7b-b586-b8de84e9c2e9 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036203 Sep 11 01:00:40.591276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 444/875] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:00:40.896348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:40.897106 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:40.903493 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:40.903973 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:40.904401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:40.905087 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:40.905577 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:40.906268 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:41.007211 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:41.007945 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:41.014127 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:41.014614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:41.015073 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:41.015771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:41.016228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:41.016649 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:41.041590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:41.049360 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d6c35464-cf9b-4047-8762-9c3a62bc8387 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1513 microversion: 2.1 time: 0.157615 Sep 11 01:00:41.049926 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 432/876] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:40 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1513 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:41.151187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:41.159138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8407708b-7d88-4b51-833a-12a2ce1dfe15 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1363 microversion: 2.1 time: 0.157366 Sep 11 01:00:41.159703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 445/877] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:41 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:00:42.008954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-e0544443-aed1-4638-9bcd-d10c13dddb38 req-90451af0-54b8-4f26-9a7f-13ba05cee734 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbd5152f-57bc-4ad1-8009-70681c4097d7", "name": "network-vif-deleted", "tag": "581e085b-9fa5-4985-97c1-4dfd99d9a05d"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:42.014406 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:42.015107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:42.022331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-e0544443-aed1-4638-9bcd-d10c13dddb38 req-90451af0-54b8-4f26-9a7f-13ba05cee734 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:42.022839 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-e0544443-aed1-4638-9bcd-d10c13dddb38 req-90451af0-54b8-4f26-9a7f-13ba05cee734 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:42.023276 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-e0544443-aed1-4638-9bcd-d10c13dddb38 req-90451af0-54b8-4f26-9a7f-13ba05cee734 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:42.034703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-e0544443-aed1-4638-9bcd-d10c13dddb38 req-90451af0-54b8-4f26-9a7f-13ba05cee734 service nova] Creating event network-vif-deleted:581e085b-9fa5-4985-97c1-4dfd99d9a05d for instance fbd5152f-57bc-4ad1-8009-70681c4097d7 on hv-cinder-maste Sep 11 01:00:42.040509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-e0544443-aed1-4638-9bcd-d10c13dddb38 req-90451af0-54b8-4f26-9a7f-13ba05cee734 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035404 Sep 11 01:00:42.041099 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 433/878] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:00:42.072117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:42.072733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:42.074660 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:42.080940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:42.081429 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:42.081863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:42.082562 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:42.083154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:42.083695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:42.152923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:42.162052 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-dbbf6582-6410-4c94-9f78-bc9755174d71 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1471 microversion: 2.1 time: 0.093803 Sep 11 01:00:42.162749 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 446/879] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:42 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1471 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:42.182689 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:42.183444 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:42.191879 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:42.192971 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:42.193477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:42.194349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:42.195236 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:42.195682 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:42.350969 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:42.360215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2b3a7b00-9aed-4f01-a57e-6e9451149860 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1363 microversion: 2.1 time: 0.181727 Sep 11 01:00:42.362188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 434/880] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:42 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1363 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:43.186356 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:43.187578 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:43.194143 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.194636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.195097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.195795 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.196304 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.196663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.231316 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.231724 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.232163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.232609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lazy-loading 'fault' on Instance uuid fbd5152f-57bc-4ad1-8009-70681c4097d7 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:00:43.281180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a2199d3f1d4bc5b1f988ae06c0d609', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de36b740b8234aca8317d5e4440decec', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:43.289388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4ba683de-7ba2-4181-831a-d6f74583c30b tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 200 len: 1309 microversion: 2.1 time: 0.106962 Sep 11 01:00:43.289973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 447/881] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:43 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 1309 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:00:43.385097 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:43.385762 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:43.392922 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.393434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.394007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.394729 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.395222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.396394 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.527952 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:43.535914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5c295495-1a35-416b-ab97-1170f3dab668 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1550 microversion: 2.1 time: 0.154876 Sep 11 01:00:43.536703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 435/882] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:43 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1550 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:43.558204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:43.558706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:43.564602 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.565097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.565537 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.566228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:43.566682 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:43.567135 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:43.658215 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-525498e4-1205-47e6-9ccd-3cc995c1581b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.103956 Sep 11 01:00:43.658837 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 448/883] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 01:00:43 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-interface => generated 285 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:00:44.313237 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:44.314134 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] [instance: fbd5152f-57bc-4ad1-8009-70681c4097d7] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:44.320563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:44.321644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:44.322328 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:44.323313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:44.323986 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:44.324626 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:44.338990 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] HTTP exception thrown: Instance fbd5152f-57bc-4ad1-8009-70681c4097d7 could not be found. Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Request method failure captured: Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 HTTP/1.1 Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTLGzgZSxnOVzz73ILtWuwtVSaWoyJCnZczekl19KlRKj086-5vrDH_l0s-CUb7Sij7KnQTOct-wZvPnQDM7z7C-_llUZYxTBaLzxdGzr1tHKHOl_oC5CVrE2nS6LK36H8F8N3pW7cV0B6fyQyb-zuDTWSddkeGBRKnrrqrsY9wWiJI Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: de36b740b8234aca8317d5e4440decec Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesSnapshotTestJSON-649966533 Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:00:44.341025 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/de36b740b8234aca8317d5e4440decec"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/de36b740b8234aca8317d5e4440decec"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/de36b740b8234aca8317d5e4440decec"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_de36b740b8234aca8317d5e4440decec"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesSnapshotTestJSON-649966533 Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: de36b740b8234aca8317d5e4440decec Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-649966533 Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesSnapshotTestJSON-649966533-project Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 78a2199d3f1d4bc5b1f988ae06c0d609 Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesSnapshotTestJSON-649966533-project Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance fbd5152f-57bc-4ad1-8009-70681c4097d7 could not be found. Sep 11 01:00:44.342549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'fbd5152f-57bc-4ad1-8009-70681c4097d7'} Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbd5152f-57bc-4ad1-8009-70681c4097d7 could not be found. Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:00:44.343403 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbd5152f-57bc-4ad1-8009-70681c4097d7 could not be found. Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbd5152f-57bc-4ad1-8009-70681c4097d7 could not be found. Sep 11 01:00:44.345266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:00:44.348085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] Returning 404 to user: Instance fbd5152f-57bc-4ad1-8009-70681c4097d7 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:00:44.348085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9aa6ee9e-6eea-469f-a0f2-45b2548250d8 tempest-VolumesSnapshotTestJSON-649966533 tempest-VolumesSnapshotTestJSON-649966533-project] 10.222.0.27 "GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7" status: 404 len: 111 microversion: 2.1 time: 0.035219 Sep 11 01:00:44.348085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 436/884] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:44 2022] GET /compute/v2.1/servers/fbd5152f-57bc-4ad1-8009-70681c4097d7 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:00:44.391168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-857433884", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:44.459257 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] validate_networks() for None {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:00:44.920933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:00:44.921368 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:00:44.921844 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:00:44.922292 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:00:44.922701 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:00:44.932076 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:44.932567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:44.933003 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:44.933736 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:44.934190 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:44.934618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:44.953961 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Getting quotas for project d99b9dcee07048b49f75a4e0172b42be. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:00:44.958811 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Getting quotas for user e0a15a4c53da4903a3f1444db955ab79 and project d99b9dcee07048b49f75a4e0172b42be. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:00:44.970282 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:00:44.972628 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:00:45.021491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-71e72ee1-cb35-449b-9136-d0aa5b3fa858 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.633838 Sep 11 01:00:45.022443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 449/885] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:00:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 636 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:00:45.034506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-033c81dd-d339-4b1d-b2da-e368b9d89db9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:45.035140 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-033c81dd-d339-4b1d-b2da-e368b9d89db9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:45.121604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-033c81dd-d339-4b1d-b2da-e368b9d89db9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:45.133796 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-033c81dd-d339-4b1d-b2da-e368b9d89db9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1214 microversion: 2.1 time: 0.101088 Sep 11 01:00:45.134751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 437/886] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:45 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1214 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:45.156817 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-ce7ab144-a718-4dc4-b631-947eb00500e7 req-155cb999-b13a-43b7-8e29-2fc1c7747f78 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0875c4c-2396-4199-bf90-c505cd4ab7f8", "tag": "92f73122-c3f2-4b96-9d29-f12911010f02"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:45.168446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ce7ab144-a718-4dc4-b631-947eb00500e7 req-155cb999-b13a-43b7-8e29-2fc1c7747f78 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:45.168871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ce7ab144-a718-4dc4-b631-947eb00500e7 req-155cb999-b13a-43b7-8e29-2fc1c7747f78 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:45.169843 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-ce7ab144-a718-4dc4-b631-947eb00500e7 req-155cb999-b13a-43b7-8e29-2fc1c7747f78 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:45.184671 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-ce7ab144-a718-4dc4-b631-947eb00500e7 req-155cb999-b13a-43b7-8e29-2fc1c7747f78 service nova] Creating event network-changed:92f73122-c3f2-4b96-9d29-f12911010f02 for instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 on hv-cinder-maste Sep 11 01:00:45.190170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-ce7ab144-a718-4dc4-b631-947eb00500e7 req-155cb999-b13a-43b7-8e29-2fc1c7747f78 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037974 Sep 11 01:00:45.191168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 450/887] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:00:46.157580 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:46.158222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:46.166346 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:46.166865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:46.167334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:46.168033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:46.168495 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:46.168919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:46.232783 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:46.241001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-30b5ecae-4c99-4f64-922a-fd7ba92101bd tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1266 microversion: 2.1 time: 0.087511 Sep 11 01:00:46.241569 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 438/888] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:46 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1266 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:47.266505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:47.267261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:47.269138 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=4 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:47.275533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:47.276049 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:47.276499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:47.277166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:47.277616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:47.278041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:47.342166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:47.349860 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3a22586f-c502-4d89-8730-f5757c582f0c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1266 microversion: 2.1 time: 0.088198 Sep 11 01:00:47.350418 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 451/889] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:47 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1266 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:48.372372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:48.372923 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:48.379352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:48.379840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:48.380313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:48.381005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:48.381457 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:48.381880 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:48.442885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:48.450420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c976f38b-1a95-4f4b-9065-488d1caa583f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1266 microversion: 2.1 time: 0.082034 Sep 11 01:00:48.451007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 439/890] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:48 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1266 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:49.473407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:49.474229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:49.480962 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:49.481448 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:49.481882 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:49.482575 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:49.483057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:49.483482 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:49.564432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:49.572625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-91c8ebe7-2346-4f7a-b592-e2e015c3624c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1272 microversion: 2.1 time: 0.103148 Sep 11 01:00:49.573201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 452/891] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:49 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1272 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:50.598094 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:50.599363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:50.605838 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:50.606328 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:50.606766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:50.607480 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:50.607936 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:50.608363 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:50.739202 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:50.746942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-df4457f3-8daa-48da-a08a-8d5b0e84ace2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.152723 Sep 11 01:00:50.747502 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 440/892] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:50 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:51.770019 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:51.770647 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:51.777476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:51.777973 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:51.778406 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:51.779347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:51.779809 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:51.780237 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:52.010055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:52.018395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-33c76f62-6f7b-4df1-b090-2278d3ee7fb9 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.252223 Sep 11 01:00:52.031395 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 453/893] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:51 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:52.515261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-4cc69c18-3f68-4aae-871f-823631e4aa20 req-2b7473ab-263c-4ade-9478-7af0edabda17 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0720ceb1-1314-4d6b-a241-4b10b97801d6", "tag": "15720af2-bce2-4ba7-8d8c-71c82a7506d7"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:52.533900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4cc69c18-3f68-4aae-871f-823631e4aa20 req-2b7473ab-263c-4ade-9478-7af0edabda17 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:52.534898 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4cc69c18-3f68-4aae-871f-823631e4aa20 req-2b7473ab-263c-4ade-9478-7af0edabda17 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:52.535588 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-4cc69c18-3f68-4aae-871f-823631e4aa20 req-2b7473ab-263c-4ade-9478-7af0edabda17 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:52.562725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-4cc69c18-3f68-4aae-871f-823631e4aa20 req-2b7473ab-263c-4ade-9478-7af0edabda17 service nova] Creating event network-changed:15720af2-bce2-4ba7-8d8c-71c82a7506d7 for instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 on hv-cinder-maste Sep 11 01:00:52.570089 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-4cc69c18-3f68-4aae-871f-823631e4aa20 req-2b7473ab-263c-4ade-9478-7af0edabda17 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059064 Sep 11 01:00:52.571527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 441/894] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:00:53.043307 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:53.043949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:53.050632 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:53.051403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:53.052019 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:53.052732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:53.053185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:53.053605 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:53.284807 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:53.293168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-792817aa-7892-485c-a7a2-08fe7ba8006f tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.254064 Sep 11 01:00:53.293804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 454/895] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:53 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:54.316797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:54.317503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:54.324573 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:54.325076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:54.325520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:54.326432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:54.326932 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:54.327488 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:54.841647 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:54.851459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c85e2d3c-6290-4337-847c-8ef7b679aa3d tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.539268 Sep 11 01:00:54.852342 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 442/896] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:54 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:55.814772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2003269827", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-975872804", "security_groups": [{"name": "tempest-secgroup-smoke-926362896"}], "block_device_mapping_v2": [{"uuid": "dd22e606-4c3a-448e-ae13-2e97c9fc54d3", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "6df3e98e-548f-496e-bd19-e33fc45eb877"}]}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:55.825375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:55.826140 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:55.875421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:55.876843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:55.879987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:55.887166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:55.888055 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:55.888500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:55.889210 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:55.889674 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:55.890107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:55.912751 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:55.955432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] validate_networks() for [('6df3e98e-548f-496e-bd19-e33fc45eb877', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:00:56.151995 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:56.164085 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2e6b0263-94f9-46e9-96b7-82a189f6ed43 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.288328 Sep 11 01:00:56.167825 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 443/897] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:55 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:00:56.423139 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:00:56.423565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:00:56.424038 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:00:56.424447 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:00:56.424857 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:00:56.434296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:56.434706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:56.435168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:56.435891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:56.436340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:56.436758 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:56.453732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for project a1368870ddbd49be857e9ac128bc684e. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:00:56.457646 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Getting quotas for user e1012f856881485cb40d916131a5d763 and project a1368870ddbd49be857e9ac128bc684e. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:00:56.514851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:00:56.516987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='dd22e606-4c3a-448e-ae13-2e97c9fc54d3',volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:00:56.655969 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a4b239ba-c298-4ca5-bf8c-c8aa7d6a979d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.844952 Sep 11 01:00:56.656600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 455/898] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:00:55 2022] POST /compute/v2.1/servers => generated 405 bytes in 846 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:00:56.668320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-873176d3-eafd-49f3-ac10-d82538c2bb41 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:56.668930 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-873176d3-eafd-49f3-ac10-d82538c2bb41 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:56.722311 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-873176d3-eafd-49f3-ac10-d82538c2bb41 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:56.730397 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-873176d3-eafd-49f3-ac10-d82538c2bb41 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1094 microversion: 2.1 time: 0.066218 Sep 11 01:00:56.730997 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 444/899] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:56 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1094 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:57.188182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:57.188804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:57.195746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:57.196233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:57.196665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:57.197360 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:57.197814 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:57.198275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:57.345249 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:57.355033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-33324626-077b-4494-9086-1c6d343ca53e tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.171186 Sep 11 01:00:57.355694 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 456/900] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:57 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:57.753566 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:57.754185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:57.761111 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:57.761595 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:57.762021 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:57.762712 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:57.763187 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:57.763613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:57.782553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-7e42ee52-d90d-43c3-b8c1-739a3cc88270 req-7960df9c-b535-4e84-b82b-acacb53973ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0720ceb1-1314-4d6b-a241-4b10b97801d6", "name": "network-vif-plugged", "status": "completed", "tag": "15720af2-bce2-4ba7-8d8c-71c82a7506d7"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:00:57.793859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7e42ee52-d90d-43c3-b8c1-739a3cc88270 req-7960df9c-b535-4e84-b82b-acacb53973ff service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:57.794319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7e42ee52-d90d-43c3-b8c1-739a3cc88270 req-7960df9c-b535-4e84-b82b-acacb53973ff service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:57.794748 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7e42ee52-d90d-43c3-b8c1-739a3cc88270 req-7960df9c-b535-4e84-b82b-acacb53973ff service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:57.806669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-7e42ee52-d90d-43c3-b8c1-739a3cc88270 req-7960df9c-b535-4e84-b82b-acacb53973ff service nova] Creating event network-vif-plugged:15720af2-bce2-4ba7-8d8c-71c82a7506d7 for instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 on hv-cinder-maste Sep 11 01:00:57.811799 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-7e42ee52-d90d-43c3-b8c1-739a3cc88270 req-7960df9c-b535-4e84-b82b-acacb53973ff service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033721 Sep 11 01:00:57.812455 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 457/901] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:00:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:00:57.834211 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:57.842185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-59210f70-3cec-49aa-abf1-7b1667c32ed4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1192 microversion: 2.1 time: 0.092613 Sep 11 01:00:57.842749 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 445/902] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:57 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1192 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:58.381879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:58.382513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:58.388930 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:58.389410 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:58.389839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:58.390524 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:58.391006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:58.391430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:58.518332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:58.521932 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=100 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:58.527032 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a5be2abc-d673-49aa-8ff9-1b732934f7d0 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.150160 Sep 11 01:00:58.527597 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 458/903] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:58 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:58.865418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:58.866081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:58.872426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:58.872910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:58.873344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:58.874033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:58.874502 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:58.874950 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:58.941265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:58.949467 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4d30c4b4-26f0-4152-ae91-c9c29d89d281 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1192 microversion: 2.1 time: 0.088007 Sep 11 01:00:58.950028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 446/904] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:58 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1192 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:59.550285 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:59.550922 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:59.557834 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:59.558312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:59.558737 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:59.559437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:59.559882 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:59.560300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:59.692582 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:00:59.700808 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-5fb5b36a-e03e-48d0-b452-b1e402bc193c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.154589 Sep 11 01:00:59.701366 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 459/905] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:59 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:00:59.973481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:00:59.974820 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:00:59.981210 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:59.981700 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:59.982136 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:00:59.982854 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:00:59.983313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:00:59.983738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:00.044335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:00.051803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8eb84cd5-7f76-4e65-9b94-44a319670aa5 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1192 microversion: 2.1 time: 0.082235 Sep 11 01:01:00.053015 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 447/906] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:00:59 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1192 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:00.723585 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:00.725077 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:00.732239 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:00.732791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:00.733291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:00.734039 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:00.734555 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:00.735092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:00.861640 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:00.869438 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-39d39f1a-647d-40f7-a35f-4305db0aa110 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1314 microversion: 2.1 time: 0.150228 Sep 11 01:01:00.869989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 460/907] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:00 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:01.075788 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:01.076558 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:01.083247 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:01.084662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:01.085395 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:01.086313 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:01.086979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:01.087410 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:01.159195 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:01.167692 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0009892d-d561-45ee-b4cf-46ae457d2633 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1210 microversion: 2.1 time: 0.096163 Sep 11 01:01:01.168261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 448/908] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:01 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1210 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:01.893155 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:01.893754 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:01.900884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:01.901375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:01.901956 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:01.902852 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:01.903492 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:01.904102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.159733 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:02.169925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-dc7a84e1-107a-41c9-9456-33e3e4e0215a tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1463 microversion: 2.1 time: 0.281155 Sep 11 01:01:02.170736 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 461/909] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:01 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1463 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:02.191284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:02.191910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:02.197464 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:02.205959 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.206593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.207075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.207843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.208502 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.208943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.227890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:02.228656 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:02.237559 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.238058 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.238717 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.241804 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.242283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.242988 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.270222 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:01:02.358163 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7b4c664f-4481-4543-bb8f-a85ac3d83d3c tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "DELETE /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 204 len: 0 microversion: 2.1 time: 0.135400 Sep 11 01:01:02.358909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 462/910] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:01:02 2022] DELETE /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 11 01:01:02.370684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:02.371057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:02.376976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.377471 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.377909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.378608 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.379111 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.379538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.509032 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:02.517765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0ccdfb94-944a-48e5-b070-a5ea8d06f345 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1277 microversion: 2.1 time: 0.330595 Sep 11 01:01:02.518372 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 449/911] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:02 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1277 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:02.619180 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:02.626940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-59baf061-a42f-4004-b518-fbe79648ef81 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1469 microversion: 2.1 time: 0.262385 Sep 11 01:01:02.627503 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 463/912] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:02 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1469 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:02.748932 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-6c22323d-a7f5-4314-96a7-dd0641eea2a5 req-fbd14d86-8097-4827-b876-37daa0b0f150 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9fa9d72f-859a-4226-a36e-d6184259b87f", "tag": "b10a8d16-c478-4d06-9202-ebcd673aec65"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:02.758803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6c22323d-a7f5-4314-96a7-dd0641eea2a5 req-fbd14d86-8097-4827-b876-37daa0b0f150 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:02.759280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6c22323d-a7f5-4314-96a7-dd0641eea2a5 req-fbd14d86-8097-4827-b876-37daa0b0f150 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:02.759709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-6c22323d-a7f5-4314-96a7-dd0641eea2a5 req-fbd14d86-8097-4827-b876-37daa0b0f150 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:02.773507 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-6c22323d-a7f5-4314-96a7-dd0641eea2a5 req-fbd14d86-8097-4827-b876-37daa0b0f150 service nova] Creating event network-changed:b10a8d16-c478-4d06-9202-ebcd673aec65 for instance 9fa9d72f-859a-4226-a36e-d6184259b87f on hv-cinder-maste Sep 11 01:01:02.778477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-6c22323d-a7f5-4314-96a7-dd0641eea2a5 req-fbd14d86-8097-4827-b876-37daa0b0f150 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033216 Sep 11 01:01:02.779088 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 450/913] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:01:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:01:03.542100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:03.542706 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:03.549070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:03.549549 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:03.550114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:03.550854 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:03.551305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:03.551723 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:03.650643 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:03.651463 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:03.658340 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:03.658863 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:03.659304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:03.660004 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:03.660464 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:03.660892 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:03.686519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:03.694417 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b6a28c42-cf01-4b42-85b1-cb81831a04e0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.156129 Sep 11 01:01:03.695012 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 464/914] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:03 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:03.793413 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:03.801200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3f32d7e8-0a28-4b75-9c5d-98b987d0da96 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1469 microversion: 2.1 time: 0.155270 Sep 11 01:01:03.801762 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 451/915] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:03 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1469 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:04.635352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-71d125d2-4f20-4165-ad56-46349a7e0b78 req-65bc7abd-9d49-4ec6-a148-8aa4838a08d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0720ceb1-1314-4d6b-a241-4b10b97801d6", "name": "network-vif-deleted", "tag": "15720af2-bce2-4ba7-8d8c-71c82a7506d7"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:04.646157 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-71d125d2-4f20-4165-ad56-46349a7e0b78 req-65bc7abd-9d49-4ec6-a148-8aa4838a08d1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:04.646962 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-71d125d2-4f20-4165-ad56-46349a7e0b78 req-65bc7abd-9d49-4ec6-a148-8aa4838a08d1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:04.647394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-71d125d2-4f20-4165-ad56-46349a7e0b78 req-65bc7abd-9d49-4ec6-a148-8aa4838a08d1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:04.669301 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-71d125d2-4f20-4165-ad56-46349a7e0b78 req-65bc7abd-9d49-4ec6-a148-8aa4838a08d1 service nova] Creating event network-vif-deleted:15720af2-bce2-4ba7-8d8c-71c82a7506d7 for instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 on hv-cinder-maste Sep 11 01:01:04.674432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-71d125d2-4f20-4165-ad56-46349a7e0b78 req-65bc7abd-9d49-4ec6-a148-8aa4838a08d1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043001 Sep 11 01:01:04.675087 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 465/916] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:01:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:01:04.717831 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:04.721130 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:04.741490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:04.742140 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:04.742673 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:04.743511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:04.747046 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:04.747841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:04.825958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:04.826657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:04.833377 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:04.833604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:04.834083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:04.834866 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:04.835322 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:04.835746 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:04.871276 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:04.871766 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:04.872198 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:04.872644 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lazy-loading 'fault' on Instance uuid 0720ceb1-1314-4d6b-a241-4b10b97801d6 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:01:04.987959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0a15a4c53da4903a3f1444db955ab79', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd99b9dcee07048b49f75a4e0172b42be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:04.996847 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1fa60eba-0772-46e9-8fc6-312737adb9a2 tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 200 len: 1304 microversion: 2.1 time: 0.175157 Sep 11 01:01:04.997408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 466/917] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:04 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 1304 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:05.063616 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:05.071419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ee42fcbc-a1c3-4563-aa49-d35789ff2aa7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.357764 Sep 11 01:01:05.071977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 452/918] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:04 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:06.019893 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:06.020430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] [instance: 0720ceb1-1314-4d6b-a241-4b10b97801d6] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:06.027457 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:06.027941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:06.028375 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:06.029065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:06.029519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:06.029946 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:06.043627 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] HTTP exception thrown: Instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 could not be found. Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Request method failure captured: Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 HTTP/1.1 Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTM6iMjEV8s4rndX5tuFEHDJSnX6DlzHNHL78RbRdIJyI2W2UAts2lT_DFJpcPcW1yI95--uTqN8IIPHVXZQVtSy4H9W7gdyLMfGR4P_WYMkLnW_2YBt0ffkrv-nvhFmkCUFCKFDcVYktkdJSGKUDL1apCaGgdT_dddDpT6hAVezFYQ Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: d99b9dcee07048b49f75a4e0172b42be Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesNegativeTest-719334876 Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:01:06.045244 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d99b9dcee07048b49f75a4e0172b42be"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d99b9dcee07048b49f75a4e0172b42be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d99b9dcee07048b49f75a4e0172b42be"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d99b9dcee07048b49f75a4e0172b42be"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesNegativeTest-719334876 Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: d99b9dcee07048b49f75a4e0172b42be Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesNegativeTest-719334876 Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesNegativeTest-719334876-project Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: e0a15a4c53da4903a3f1444db955ab79 Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesNegativeTest-719334876-project Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 could not be found. Sep 11 01:01:06.047016 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '0720ceb1-1314-4d6b-a241-4b10b97801d6'} Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 could not be found. Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:01:06.048056 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 could not be found. Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 could not be found. Sep 11 01:01:06.049629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:01:06.051103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] Returning 404 to user: Instance 0720ceb1-1314-4d6b-a241-4b10b97801d6 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:01:06.051103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d29b5bf2-05dd-4c6f-8f71-aabf74aa5cdf tempest-VolumesNegativeTest-719334876 tempest-VolumesNegativeTest-719334876-project] 10.222.0.27 "GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6" status: 404 len: 111 microversion: 2.1 time: 0.030730 Sep 11 01:01:06.051103 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 467/919] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:06 2022] GET /compute/v2.1/servers/0720ceb1-1314-4d6b-a241-4b10b97801d6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:01:06.093300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:06.093888 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:06.100830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:06.101318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:06.101752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:06.102449 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:06.102916 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:06.103343 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:06.242683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:06.250955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-79c446bb-7edb-4cee-9948-2d5e6cab0f31 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.161396 Sep 11 01:01:06.251527 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 453/920] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:06 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:07.274577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:07.275212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:07.277167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:07.277929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:07.284738 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:07.285225 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:07.285659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:07.286544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:07.287024 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:07.287439 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:07.426657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:07.435268 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9d1890b8-cba6-4a48-a5ab-80d2394cc4eb tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.165695 Sep 11 01:01:07.435831 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 468/921] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:07 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:08.458725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:08.459374 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:08.461309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:08.467456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:08.467943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:08.468381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:08.469080 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:08.469534 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:08.469962 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:08.610424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:08.619282 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-56357a5b-ed44-40d4-a1d4-8a97dba47b0b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.164424 Sep 11 01:01:08.619904 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 454/922] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:08 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:09.642771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:09.643326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:09.650017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:09.650522 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:09.650990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:09.651672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:09.652120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:09.652541 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:09.779372 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:09.787851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-175deb6a-4a9b-467f-8646-47b96ee1163e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.149209 Sep 11 01:01:09.788404 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 469/923] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:09 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:10.811900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:10.812729 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:10.820603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:10.820767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:10.821243 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:10.821976 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:10.822430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:10.822889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:10.958261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:10.966203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d8828b58-c13b-48de-b4ae-351ee07de60b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.158755 Sep 11 01:01:10.966901 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 455/924] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:10 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:11.989806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:11.990512 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:11.997098 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:11.997522 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:11.997952 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:11.998859 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:11.999312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:11.999739 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:12.164473 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:12.172269 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-77520e45-f177-4f69-8c56-d19ecb2111e7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.186520 Sep 11 01:01:12.172839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 470/925] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:11 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:13.197221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:13.197867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:13.204320 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:13.204809 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:13.205250 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:13.205940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:13.206402 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:13.206852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:13.341726 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:13.349613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6d7628cb-638d-4a6e-88fa-7ec6990fb1c8 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.156317 Sep 11 01:01:13.350184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 456/926] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:13 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:14.373292 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:14.373842 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:14.380277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:14.380759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:14.381195 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:14.381879 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:14.382332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:14.382753 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:14.511201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:14.518763 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d2aead9b-722b-4b86-99c6-61b3ba6a6819 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.149433 Sep 11 01:01:14.519347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 471/927] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:14 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:15.542280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:15.542911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:15.549284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:15.549766 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:15.550197 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:15.550910 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:15.552067 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:15.552477 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:15.686186 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:15.694304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8cece671-6072-470e-adbc-5803070c4dc0 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.156102 Sep 11 01:01:15.694815 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 457/928] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:15 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:16.716889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:16.717508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:16.724134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:16.724616 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:16.725051 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:16.725743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:16.726204 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:16.726628 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:16.853927 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:16.861974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-95382443-6e9e-4e05-967f-76eef750315d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.148872 Sep 11 01:01:16.862565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 472/929] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:16 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:17.886144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:17.886807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:17.893199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:17.893687 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:17.894124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:17.894913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:17.896051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:17.896532 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:18.018781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-bb0f719b-d2ad-486e-8429-eb908d912bbf req-754e5246-1bed-4fa5-b893-2358b8402231 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9d72f-859a-4226-a36e-d6184259b87f", "name": "network-vif-plugged", "status": "completed", "tag": "b10a8d16-c478-4d06-9202-ebcd673aec65"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:18.026284 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:18.034611 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bb0f719b-d2ad-486e-8429-eb908d912bbf req-754e5246-1bed-4fa5-b893-2358b8402231 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:18.035121 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bb0f719b-d2ad-486e-8429-eb908d912bbf req-754e5246-1bed-4fa5-b893-2358b8402231 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:18.035552 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-bb0f719b-d2ad-486e-8429-eb908d912bbf req-754e5246-1bed-4fa5-b893-2358b8402231 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:18.041407 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:18.048452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-bb0f719b-d2ad-486e-8429-eb908d912bbf req-754e5246-1bed-4fa5-b893-2358b8402231 service nova] Creating event network-vif-plugged:b10a8d16-c478-4d06-9202-ebcd673aec65 for instance 9fa9d72f-859a-4226-a36e-d6184259b87f on hv-cinder-maste Sep 11 01:01:18.049347 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4f205164-86e9-4376-a84f-48543b61241f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.167090 Sep 11 01:01:18.050144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 458/930] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:17 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:18.053987 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-bb0f719b-d2ad-486e-8429-eb908d912bbf req-754e5246-1bed-4fa5-b893-2358b8402231 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039098 Sep 11 01:01:18.054766 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 473/931] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:01:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:01:19.072769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:19.073388 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:19.079950 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:19.080444 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:19.080885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:19.081616 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:19.082076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:19.083104 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:19.215221 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:19.222862 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b889da7f-390f-473b-957f-6829f17a6224 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1265 microversion: 2.1 time: 0.153962 Sep 11 01:01:19.223536 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 459/932] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:19 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:20.246332 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:20.247079 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:20.255359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:20.256025 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:20.256461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:20.257149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:20.257604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:20.258026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:20.388086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:20.396090 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d9cb8136-21ee-4fa1-81c5-e87da6d9cb38 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1451 microversion: 2.1 time: 0.154110 Sep 11 01:01:20.396652 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 474/933] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:20 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1451 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:20.418840 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:20.419463 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:20.425988 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:20.427279 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:20.427725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:20.428422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:20.428885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:20.429315 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:20.560222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:20.568782 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b6d1d513-e7b7-4992-ad59-d9fd7b5441e4 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1451 microversion: 2.1 time: 0.153922 Sep 11 01:01:20.569361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 460/934] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:20 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1451 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:22.145991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-c15df33d-671d-4aef-b6fa-919294d625f5 req-6e73a0f4-03a5-4f72-8e7b-dafa857a8df0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9fa9d72f-859a-4226-a36e-d6184259b87f", "tag": "b10a8d16-c478-4d06-9202-ebcd673aec65"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:22.155721 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-c15df33d-671d-4aef-b6fa-919294d625f5 req-6e73a0f4-03a5-4f72-8e7b-dafa857a8df0 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:22.156301 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-c15df33d-671d-4aef-b6fa-919294d625f5 req-6e73a0f4-03a5-4f72-8e7b-dafa857a8df0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:22.156791 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-c15df33d-671d-4aef-b6fa-919294d625f5 req-6e73a0f4-03a5-4f72-8e7b-dafa857a8df0 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:22.167657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-c15df33d-671d-4aef-b6fa-919294d625f5 req-6e73a0f4-03a5-4f72-8e7b-dafa857a8df0 service nova] Creating event network-changed:b10a8d16-c478-4d06-9202-ebcd673aec65 for instance 9fa9d72f-859a-4226-a36e-d6184259b87f on hv-cinder-maste Sep 11 01:01:22.173097 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-c15df33d-671d-4aef-b6fa-919294d625f5 req-6e73a0f4-03a5-4f72-8e7b-dafa857a8df0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030934 Sep 11 01:01:22.173907 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 475/935] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:01:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:01:28.943141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:28.943741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:28.951612 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:28.952095 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:28.952525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:28.953206 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:28.953654 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:28.954073 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:28.956401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=95 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:28.979458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7bd8913e-09f5-4ec5-9f48-c3ad7727598b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/ips" status: 200 len: 147 microversion: 2.1 time: 0.041303 Sep 11 01:01:28.980011 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 461/936] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 01:01:28 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/ips => generated 147 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:01:30.953254 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "49845893-43ac-476d-b05f-7180ec205d9e"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:30.955713 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:30.963031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:30.963511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:30.963940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:30.964627 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:30.965080 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:30.965499 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:30.967925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:31.372212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-facba1aa-74e6-4ff1-9e02-b2f1d41d2a67 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.423062 Sep 11 01:01:31.372772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 476/937] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 01:01:30 2022] POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments => generated 194 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:01:38.030505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1c8561bc-c6e0-49a5-b1ed-4c797bbf3811 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:38.033065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=26 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:38.039578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1c8561bc-c6e0-49a5-b1ed-4c797bbf3811 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:38.040075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1c8561bc-c6e0-49a5-b1ed-4c797bbf3811 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:38.040551 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1c8561bc-c6e0-49a5-b1ed-4c797bbf3811 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:38.102096 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1c8561bc-c6e0-49a5-b1ed-4c797bbf3811 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Going to try to stop instance {{(pid=126979) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 11 01:01:38.147655 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1c8561bc-c6e0-49a5-b1ed-4c797bbf3811 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action" status: 202 len: 0 microversion: 2.1 time: 0.121064 Sep 11 01:01:38.148259 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 462/938] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 01:01:38 2022] POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action => generated 0 bytes in 123 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:01:38.157691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:38.158394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:38.164533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:38.165014 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:38.165476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:38.166146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:38.166595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:38.167057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:38.296669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:38.305538 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d49d0358-2c62-40b2-967e-4a5015fd9083 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.152920 Sep 11 01:01:38.306094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 477/939] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:38 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:39.330115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:39.330851 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:39.337079 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:39.337570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:39.338010 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:39.338707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:39.339188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:39.339616 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:39.467841 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:39.476637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-97b47803-8f93-4c02-9a56-2d5c036caaa9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.151369 Sep 11 01:01:39.477196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 463/940] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:39 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:40.500218 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:40.500823 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:40.507423 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:40.507902 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:40.508330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:40.509010 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:40.509456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:40.509875 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:40.639655 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:40.648863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-632586b9-ce04-4e01-8ddb-ab2b0b008ed5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.152570 Sep 11 01:01:40.649418 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 478/941] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:40 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:41.672806 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:41.673437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:41.680579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:41.681071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:41.681509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:41.682205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:41.682702 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:41.683118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:41.811765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:41.820245 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2381c1e2-1369-4309-91ca-e6987b748370 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.151347 Sep 11 01:01:41.820803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 464/942] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:41 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:42.844058 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:42.844772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:42.851485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:42.851976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:42.852407 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:42.853093 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:42.853544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:42.853964 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:42.991600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:42.999868 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e737cfbf-2d73-4173-a2f3-58dc35eb5bc9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.159845 Sep 11 01:01:43.000427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 479/943] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:42 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:44.024940 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:44.025570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:44.032185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:44.032668 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:44.033100 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:44.033783 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:44.034233 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:44.034650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:44.163235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:44.171399 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-eaaa2623-a003-4570-863d-0a26c796c524 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.151264 Sep 11 01:01:44.171953 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 465/944] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:44 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:45.194466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:45.195102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:45.201497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:45.201976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:45.202419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:45.203131 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:45.203586 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:45.204007 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:45.330110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:45.339185 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cd787d1e-d54b-42b6-ad3c-c72c95f57238 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.148615 Sep 11 01:01:45.339742 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 480/945] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:45 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:46.367322 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:46.367905 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:46.374265 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:46.374750 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:46.375223 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:46.375912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:46.376370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:46.376793 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:46.504518 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:46.514448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0d55c6ce-c1a7-4351-96d9-f8d8bdb75bfa tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.155757 Sep 11 01:01:46.515050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 466/946] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:46 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:47.537968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:47.538588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:47.544929 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:47.545414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:47.545847 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:47.546533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:47.547011 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:47.547433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:47.672485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:47.681072 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-79516187-b8c6-4771-9dd4-b91b771275ec tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.147055 Sep 11 01:01:47.681628 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 481/947] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:47 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:48.704327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:48.704939 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:48.711770 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:48.712251 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:48.712681 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:48.713361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:48.713810 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:48.714235 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:48.845144 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:48.854456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f8a1720f-bd90-427b-92ee-c2632c6ec1d8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.154050 Sep 11 01:01:48.855052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 467/948] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:48 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:49.884798 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:49.885419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:49.891822 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:49.892309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:49.892743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:49.893430 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:49.893880 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:49.894300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:50.022732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:50.031168 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d56ae33f-fa6c-4d43-8253-501f688616ff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.150241 Sep 11 01:01:50.031728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 482/949] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:49 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:01:51.055448 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:51.056185 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:51.062632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:51.063157 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:51.063601 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:51.064294 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:51.064783 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:51.065213 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:51.198326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:51.206413 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b965b07d-983f-47c9-8f30-b3c374891de1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.155529 Sep 11 01:01:51.206999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 468/950] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:51 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:52.230077 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:52.230684 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:52.233063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=20 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:52.238234 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:52.238994 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:52.239445 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:52.240132 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:52.240580 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:52.240997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:52.370379 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:52.378433 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-052f28c4-23bf-4bed-8202-2d24c0720d94 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.152212 Sep 11 01:01:52.379004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 483/951] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:52 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:53.402630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:53.403219 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:53.409683 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:53.410162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:53.410593 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:53.411389 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:53.412065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:53.412495 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:53.539536 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:53.547644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-04b6f6e5-1372-41b4-9ef0-3f640fa9ca87 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1725 microversion: 2.1 time: 0.148976 Sep 11 01:01:53.548205 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 469/952] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:53 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1725 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:54.571803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:54.572431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:54.579263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:54.579759 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:54.580184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:54.580877 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:54.581329 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:54.581750 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:54.719368 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:54.728336 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-99082b82-3354-47d1-95c5-4e7cd41eb708 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1702 microversion: 2.1 time: 0.160671 Sep 11 01:01:54.728971 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 484/953] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:54 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1702 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:54.752503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c33b83bf-6d61-4e27-a02b-f223aec98486 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:01:54.758861 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33b83bf-6d61-4e27-a02b-f223aec98486 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:54.759359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33b83bf-6d61-4e27-a02b-f223aec98486 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:54.759797 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c33b83bf-6d61-4e27-a02b-f223aec98486 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:54.783536 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c33b83bf-6d61-4e27-a02b-f223aec98486 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Going to try to start instance {{(pid=126979) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 11 01:01:54.827793 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c33b83bf-6d61-4e27-a02b-f223aec98486 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action" status: 202 len: 0 microversion: 2.1 time: 0.079262 Sep 11 01:01:54.828332 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 470/954] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 01:01:54 2022] POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:01:54.838178 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:54.838841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:54.845982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:54.846653 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:54.847256 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:54.848167 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:54.849651 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:54.850220 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:55.023317 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:55.032034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-86cb421c-de80-419e-bef7-9ee12976e6a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1711 microversion: 2.1 time: 0.199040 Sep 11 01:01:55.032612 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 485/955] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:54 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1711 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:56.058114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:56.058824 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:56.064946 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:56.065434 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:56.065863 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:56.066546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:56.067022 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:56.067599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:56.199452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:56.207509 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4d640180-5b0b-4a50-bdf0-d51901278f34 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1711 microversion: 2.1 time: 0.154090 Sep 11 01:01:56.208101 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 471/956] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:56 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1711 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:01:57.230960 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:01:57.231573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:01:57.237974 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:57.238484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:57.238933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:57.240444 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:01:57.240898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:01:57.241320 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:01:57.369805 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:01:57.378034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e60bea3f-b9e7-4fc5-b1f8-27007f405a2c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1715 microversion: 2.1 time: 0.151000 Sep 11 01:01:57.378595 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 486/957] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:01:57 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1715 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:29.846895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:29.847511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:29.849590 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=19 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.850271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.856220 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:29.856708 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:29.857145 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:29.857835 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:29.858288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:29.858713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:29.939052 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lazy-loading 'services' on Instance uuid b0875c4c-2396-4199-bf90-c505cd4ab7f8 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:02:30.011543 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0c603121-5fe0-41cb-a8c5-7aaa7f0876f8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments/49845893-43ac-476d-b05f-7180ec205d9e" status: 202 len: 0 microversion: 2.1 time: 0.168486 Sep 11 01:02:30.012203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 472/958] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 01:02:29 2022] DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments/49845893-43ac-476d-b05f-7180ec205d9e => generated 0 bytes in 170 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:02:31.131300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7cba2eaf-98dc-4f11-ab11-cead938746a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:02:31.133889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=7 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:31.134539 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:31.140169 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cba2eaf-98dc-4f11-ab11-cead938746a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:31.140654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cba2eaf-98dc-4f11-ab11-cead938746a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:31.141083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7cba2eaf-98dc-4f11-ab11-cead938746a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:31.201054 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7cba2eaf-98dc-4f11-ab11-cead938746a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Going to try to stop instance {{(pid=126981) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 11 01:02:31.242341 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7cba2eaf-98dc-4f11-ab11-cead938746a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action" status: 202 len: 0 microversion: 2.1 time: 0.114913 Sep 11 01:02:31.242944 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 487/959] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 01:02:31 2022] POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:02:31.251709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:31.252412 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:31.260082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:31.261036 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:31.261036 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:31.261697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:31.262148 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:31.262618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:31.397542 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:31.406386 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-435a091c-c6fc-4af0-ae1f-e92ee6f08e0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.158955 Sep 11 01:02:31.406984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 473/960] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:31 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:02:32.431157 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:32.431773 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:32.437870 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:32.438354 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:32.438811 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:32.439497 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:32.439948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:32.440371 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:32.570872 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:32.578451 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2109b3b4-cde6-4c95-866c-b4e00d71f188 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.152675 Sep 11 01:02:32.579035 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 488/961] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:32 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:33.602304 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:33.602942 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:33.609348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:33.609834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:33.610268 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:33.610980 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:33.611432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:33.611891 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:33.741091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:33.750141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f5c2737b-c57f-4493-9003-185c7e610d63 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.152220 Sep 11 01:02:33.750743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 474/962] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:33 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:34.772830 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:34.773474 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:34.781385 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:34.782069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:34.782663 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:34.783470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:34.784108 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:34.784696 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:34.916179 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:34.923806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a221a6e6-d80f-4ee0-b75e-fa37cd8b9059 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.154833 Sep 11 01:02:34.924364 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 489/963] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:34 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:35.947759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:35.948375 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:35.954773 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:35.955306 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:35.955744 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:35.956437 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:35.956894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:35.957318 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:36.084759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:36.092554 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-952641b4-333d-4d1c-96c7-0155db47cf19 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.149221 Sep 11 01:02:36.093128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 475/964] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:35 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:37.115751 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:37.116964 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:37.123604 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:37.124082 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:37.124542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:37.125228 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:37.125689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:37.126111 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:37.430743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:37.439201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-da09f6cc-a671-439b-a816-ec3f46211ad4 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.327334 Sep 11 01:02:37.439762 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 490/965] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:37 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:38.461401 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:38.462010 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:38.468400 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:38.468883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:38.469319 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:38.470006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:38.470458 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:38.470908 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:38.653495 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:38.661362 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-54a05b4d-8f34-4ec2-abcf-e7e690c2be05 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.203850 Sep 11 01:02:38.661920 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 476/966] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:38 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:39.685977 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:39.686606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:39.692823 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:39.693312 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:39.693747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:39.695100 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:39.695675 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:39.696102 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:39.825084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:39.834321 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-24940279-56ef-4600-a3c9-ab27f4424191 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.152397 Sep 11 01:02:39.834911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 491/967] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:39 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:40.857001 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:40.857637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:40.864118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:40.864603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:40.865037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:40.865723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:40.866178 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:40.866675 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:40.996191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:41.003937 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-be1f83eb-cd68-4dc1-ba20-93c1facdcd45 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1679 microversion: 2.1 time: 0.150838 Sep 11 01:02:41.004503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 477/968] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:40 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1679 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:42.029041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:42.029723 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:42.036569 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:42.037047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:42.037476 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:42.038155 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:42.039508 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:42.039976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:42.176465 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:42.184748 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-06f7ef63-280b-48cd-87af-4ffe0c98e2a1 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1656 microversion: 2.1 time: 0.160025 Sep 11 01:02:42.185305 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 492/969] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:42 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1656 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:42.207586 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-90177a7b-6cb9-4655-9227-3b630891285d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:02:42.214394 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90177a7b-6cb9-4655-9227-3b630891285d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:42.214919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90177a7b-6cb9-4655-9227-3b630891285d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:42.215351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-90177a7b-6cb9-4655-9227-3b630891285d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:42.245869 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-90177a7b-6cb9-4655-9227-3b630891285d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Going to try to start instance {{(pid=126979) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 11 01:02:42.290656 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-90177a7b-6cb9-4655-9227-3b630891285d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action" status: 202 len: 0 microversion: 2.1 time: 0.086736 Sep 11 01:02:42.291327 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 478/970] 10.222.0.27 () {60 vars in 1278 bytes} [Sun Sep 11 01:02:42 2022] POST /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:02:42.299691 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:42.300302 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:42.307314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:42.307802 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:42.308347 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:42.309972 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:42.310431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:42.310889 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:42.475976 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:42.484042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-520b515a-c286-4f17-a9d5-a26d1c6e76e3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1665 microversion: 2.1 time: 0.188255 Sep 11 01:02:42.484618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 493/971] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:42 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1665 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:43.508826 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:43.509535 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:43.515913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:43.516408 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:43.516848 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:43.517543 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:43.517999 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:43.518427 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:43.645693 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:43.653973 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-8f45d6ea-f24b-452f-8c2a-2a2f54a190e8 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1665 microversion: 2.1 time: 0.150175 Sep 11 01:02:43.654563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 479/972] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:43 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1665 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:44.678084 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:44.678625 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:44.685189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:44.685669 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:44.686099 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:44.686806 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:44.687255 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:44.687676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:44.815735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:44.824212 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3e674fc4-be23-4f83-bba4-715ba35012fe tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1669 microversion: 2.1 time: 0.150099 Sep 11 01:02:44.824771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 494/973] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:44 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1669 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:52.008050 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:52.009424 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:52.009821 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-747bedaf-6ee7-4a8b-b98b-c219894f02a8 req-88283599-cbd8-41f1-a957-549adc3c7e3c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9fa9d72f-859a-4226-a36e-d6184259b87f", "tag": "b10a8d16-c478-4d06-9202-ebcd673aec65"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:02:52.013132 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 11 01:02:52.013271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Sun Sep 11 01:02:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f (ip 10.222.0.27) !!! Sep 11 01:02:52.021680 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-747bedaf-6ee7-4a8b-b98b-c219894f02a8 req-88283599-cbd8-41f1-a957-549adc3c7e3c service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:52.022483 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-747bedaf-6ee7-4a8b-b98b-c219894f02a8 req-88283599-cbd8-41f1-a957-549adc3c7e3c service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:52.023400 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-747bedaf-6ee7-4a8b-b98b-c219894f02a8 req-88283599-cbd8-41f1-a957-549adc3c7e3c service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:52.030618 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:52.031142 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:52.031577 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:52.032277 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:52.038907 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:52.039451 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:52.042069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-747bedaf-6ee7-4a8b-b98b-c219894f02a8 req-88283599-cbd8-41f1-a957-549adc3c7e3c service nova] Creating event network-changed:b10a8d16-c478-4d06-9202-ebcd673aec65 for instance 9fa9d72f-859a-4226-a36e-d6184259b87f on hv-cinder-maste Sep 11 01:02:52.047206 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-747bedaf-6ee7-4a8b-b98b-c219894f02a8 req-88283599-cbd8-41f1-a957-549adc3c7e3c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041819 Sep 11 01:02:52.047794 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 495/974] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:02:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:02:52.068634 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:02:52.150305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-abf2a8a6-0bfd-45a2-afb4-83bedb5c4439 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "DELETE /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 204 len: 0 microversion: 2.1 time: 0.146656 Sep 11 01:02:52.151043 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 480/975] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:02:52 2022] DELETE /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 01:02:52.160041 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:52.160740 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:52.166898 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:52.167381 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:52.167823 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:52.168512 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:52.168967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:52.169388 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:52.300190 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:52.324923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-fa671f6d-e08e-40f0-8c62-33b327c2ef34 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1575 microversion: 2.1 time: 0.165448 Sep 11 01:02:52.334201 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 496/976] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:52 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1575 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:02:53.350007 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:53.350710 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:53.357663 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:53.358143 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:53.358579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:53.359316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:53.359767 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:53.360188 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:53.521933 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:53.531267 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f7fc65fe-560d-4463-b838-c4e8d1ff3db3 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1575 microversion: 2.1 time: 0.185440 Sep 11 01:02:53.531833 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 481/977] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:53 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1575 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:02:54.555498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:54.556280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:54.563831 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:54.564314 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:54.564779 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:54.565484 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:54.565909 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:54.566370 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:54.633843 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:54.645943 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-696d00a4-e60a-4674-85ca-b6b81f2ac70a tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 200 len: 1390 microversion: 2.1 time: 0.093900 Sep 11 01:02:54.646635 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 497/978] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:54 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 1390 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:02:54.690741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-686c0393-89b2-4685-a4ca-b4beb30aad43 req-e15a5cd3-e06c-47a4-abfb-b05fe9f764fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9d72f-859a-4226-a36e-d6184259b87f", "name": "network-vif-deleted", "tag": "b10a8d16-c478-4d06-9202-ebcd673aec65"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:02:54.701567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-686c0393-89b2-4685-a4ca-b4beb30aad43 req-e15a5cd3-e06c-47a4-abfb-b05fe9f764fd service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:54.702143 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-686c0393-89b2-4685-a4ca-b4beb30aad43 req-e15a5cd3-e06c-47a4-abfb-b05fe9f764fd service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:54.702638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-686c0393-89b2-4685-a4ca-b4beb30aad43 req-e15a5cd3-e06c-47a4-abfb-b05fe9f764fd service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:54.714900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-686c0393-89b2-4685-a4ca-b4beb30aad43 req-e15a5cd3-e06c-47a4-abfb-b05fe9f764fd service nova] Creating event network-vif-deleted:b10a8d16-c478-4d06-9202-ebcd673aec65 for instance 9fa9d72f-859a-4226-a36e-d6184259b87f on hv-cinder-maste Sep 11 01:02:54.720063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-686c0393-89b2-4685-a4ca-b4beb30aad43 req-e15a5cd3-e06c-47a4-abfb-b05fe9f764fd service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034616 Sep 11 01:02:54.720660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 482/979] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:02:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:02:55.670070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:55.670832 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 9fa9d72f-859a-4226-a36e-d6184259b87f] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:55.677251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:55.677735 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:55.678173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:55.678979 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:55.679438 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:55.679863 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:55.693036 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] HTTP exception thrown: Instance 9fa9d72f-859a-4226-a36e-d6184259b87f could not be found. Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Request method failure captured: Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f HTTP/1.1 Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTHw8BzyvxS9fdqb226qp72rJVPqsBknMqx-aQjJYwTnPCIl2mNNo1zdJdKfwd7rEQQXjZPG3rPu9YXC3e4Bo9Vda4P2ZVOndGogkzl1WxCa_0jqx5OE6h7WOt2I6ehPlVsDGEtKClOzhvDOMYtpYt6zh2TZKLzaYnX83PrP7WWBE48 Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:02:55.694677 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-TestVolumeBootPattern-40689968 Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: e1012f856881485cb40d916131a5d763 Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 9fa9d72f-859a-4226-a36e-d6184259b87f could not be found. Sep 11 01:02:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '9fa9d72f-859a-4226-a36e-d6184259b87f'} Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9fa9d72f-859a-4226-a36e-d6184259b87f could not be found. Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:02:55.697657 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9fa9d72f-859a-4226-a36e-d6184259b87f could not be found. Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9fa9d72f-859a-4226-a36e-d6184259b87f could not be found. Sep 11 01:02:55.699315 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:02:55.700878 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Returning 404 to user: Instance 9fa9d72f-859a-4226-a36e-d6184259b87f could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:02:55.700878 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c096b735-76e2-493c-87e1-077927d10b4f tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f" status: 404 len: 111 microversion: 2.1 time: 0.030496 Sep 11 01:02:55.700878 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 498/980] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:55 2022] GET /compute/v2.1/servers/9fa9d72f-859a-4226-a36e-d6184259b87f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:02:59.003637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:59.004369 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:59.006519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-6511c06d-1f18-47e4-a489-0a58a8b2fe29 req-8370f4bd-3383-4d7a-a3f0-fa6ec911fd64 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "459cb8ce-dc67-4fd5-a7ba-23b228fd415e", "tag": "bc9b80b7-364d-4952-97b3-277a78d5fa01"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:02:59.010968 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:59.011452 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:59.012129 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:59.012828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:59.013280 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:59.013696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:59.019330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6511c06d-1f18-47e4-a489-0a58a8b2fe29 req-8370f4bd-3383-4d7a-a3f0-fa6ec911fd64 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:59.019820 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6511c06d-1f18-47e4-a489-0a58a8b2fe29 req-8370f4bd-3383-4d7a-a3f0-fa6ec911fd64 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:59.020247 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-6511c06d-1f18-47e4-a489-0a58a8b2fe29 req-8370f4bd-3383-4d7a-a3f0-fa6ec911fd64 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:59.032125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-6511c06d-1f18-47e4-a489-0a58a8b2fe29 req-8370f4bd-3383-4d7a-a3f0-fa6ec911fd64 service nova] Creating event network-changed:bc9b80b7-364d-4952-97b3-277a78d5fa01 for instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e on hv-cinder-maste Sep 11 01:02:59.037596 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-6511c06d-1f18-47e4-a489-0a58a8b2fe29 req-8370f4bd-3383-4d7a-a3f0-fa6ec911fd64 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036218 Sep 11 01:02:59.038182 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 499/981] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:02:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:02:59.039986 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:02:59.126092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6d945345-e127-48b6-9db7-cf6c897285ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "DELETE /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 204 len: 0 microversion: 2.1 time: 0.126956 Sep 11 01:02:59.127563 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 483/982] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:02:58 2022] DELETE /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 01:02:59.138174 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:02:59.138919 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:02:59.146205 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:59.146533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:59.147020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:59.147828 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:02:59.148162 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:02:59.148588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:02:59.284940 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:02:59.292948 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7d5affad-18c5-4268-b500-92385ff2b2ed tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1575 microversion: 2.1 time: 0.160774 Sep 11 01:02:59.293515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 500/983] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:02:59 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:00.317508 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:00.318383 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:00.320828 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:00.326375 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:00.326973 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:00.327481 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:00.328182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:00.328638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:00.329059 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:00.334538 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=4,nova_cell1:UPDATE=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:00.467201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:00.474895 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ab310859-dd76-4a07-ac06-e1641c87100d tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1575 microversion: 2.1 time: 0.161665 Sep 11 01:03:00.475460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 484/984] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:00 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1575 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:01.326348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-f3685e44-3ad4-4666-8305-87774a19850c req-e82555ca-8467-4bf1-a80b-37e6551758c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "459cb8ce-dc67-4fd5-a7ba-23b228fd415e", "name": "network-vif-deleted", "tag": "bc9b80b7-364d-4952-97b3-277a78d5fa01"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:01.332873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=29 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:01.340143 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f3685e44-3ad4-4666-8305-87774a19850c req-e82555ca-8467-4bf1-a80b-37e6551758c2 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:01.340845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f3685e44-3ad4-4666-8305-87774a19850c req-e82555ca-8467-4bf1-a80b-37e6551758c2 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:01.341274 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-f3685e44-3ad4-4666-8305-87774a19850c req-e82555ca-8467-4bf1-a80b-37e6551758c2 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:01.344078 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:01.354591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-f3685e44-3ad4-4666-8305-87774a19850c req-e82555ca-8467-4bf1-a80b-37e6551758c2 service nova] Creating event network-vif-deleted:bc9b80b7-364d-4952-97b3-277a78d5fa01 for instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e on hv-cinder-maste Sep 11 01:03:01.359588 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-f3685e44-3ad4-4666-8305-87774a19850c req-e82555ca-8467-4bf1-a80b-37e6551758c2 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038087 Sep 11 01:03:01.360169 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 501/985] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:03:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:03:01.498699 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:01.499264 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:01.505798 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:01.506281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:01.506716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:01.507423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:01.507874 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:01.508292 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:01.574594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1012f856881485cb40d916131a5d763', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1368870ddbd49be857e9ac128bc684e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:01.582554 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b3589650-d43c-412d-822d-a5aa7372e9ab tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 200 len: 1224 microversion: 2.1 time: 0.088325 Sep 11 01:03:01.583450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 485/986] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:01 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:02.613363 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:02.614063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 459cb8ce-dc67-4fd5-a7ba-23b228fd415e] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:02.620672 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:02.621132 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:02.621558 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:02.622242 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:02.622688 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:02.623137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:02.636816 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] HTTP exception thrown: Instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e could not be found. Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Request method failure captured: Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e HTTP/1.1 Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTHw8BzyvxS9fdqb226qp72rJVPqsBknMqx-aQjJYwTnPCIl2mNNo1zdJdKfwd7rEQQXjZPG3rPu9YXC3e4Bo9Vda4P2ZVOndGogkzl1WxCa_0jqx5OE6h7WOt2I6ehPlVsDGEtKClOzhvDOMYtpYt6zh2TZKLzaYnX83PrP7WWBE48 Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:03:02.638399 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: e1012f856881485cb40d916131a5d763 Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e could not be found. Sep 11 01:03:02.640092 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '459cb8ce-dc67-4fd5-a7ba-23b228fd415e'} Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e could not be found. Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:03:02.641181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e could not be found. Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e could not be found. Sep 11 01:03:02.642747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:02.644446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Returning 404 to user: Instance 459cb8ce-dc67-4fd5-a7ba-23b228fd415e could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:03:02.644446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-73293410-eb54-4067-8b5c-ec91b5c2140b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e" status: 404 len: 111 microversion: 2.1 time: 0.031872 Sep 11 01:03:02.644446 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 502/987] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:02 2022] GET /compute/v2.1/servers/459cb8ce-dc67-4fd5-a7ba-23b228fd415e => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:03:03.069979 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:03.070503 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:03.077696 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:03.078184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:03.078620 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:03.079334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:03.079776 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:03.080203 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:03.093650 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] HTTP exception thrown: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Request method failure captured: Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac HTTP/1.1 Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTHw8BzyvxS9fdqb226qp72rJVPqsBknMqx-aQjJYwTnPCIl2mNNo1zdJdKfwd7rEQQXjZPG3rPu9YXC3e4Bo9Vda4P2ZVOndGogkzl1WxCa_0jqx5OE6h7WOt2I6ehPlVsDGEtKClOzhvDOMYtpYt6zh2TZKLzaYnX83PrP7WWBE48 Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:03:03.095334 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: e1012f856881485cb40d916131a5d763 Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.096987 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '984863bd-b3a1-4546-8b36-720ba324d7ac'} Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:03:03.098128 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.099511 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.101489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Returning 404 to user: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:03:03.101489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-880829e6-2bd8-48c6-8fd7-b9016f765cc7 tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "DELETE /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 404 len: 111 microversion: 2.1 time: 0.030667 Sep 11 01:03:03.101489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 486/988] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:03:03 2022] DELETE /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:03:03.107042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:03.107653 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [instance: 984863bd-b3a1-4546-8b36-720ba324d7ac] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:03.113810 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:03.114287 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:03.114714 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:03.115445 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:03.115901 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:03.116324 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:03.130165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] HTTP exception thrown: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Request method failure captured: Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac HTTP/1.1 Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTHw8BzyvxS9fdqb226qp72rJVPqsBknMqx-aQjJYwTnPCIl2mNNo1zdJdKfwd7rEQQXjZPG3rPu9YXC3e4Bo9Vda4P2ZVOndGogkzl1WxCa_0jqx5OE6h7WOt2I6ehPlVsDGEtKClOzhvDOMYtpYt6zh2TZKLzaYnX83PrP7WWBE48 Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:03:03.131685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_a1368870ddbd49be857e9ac128bc684e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: a1368870ddbd49be857e9ac128bc684e Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-TestVolumeBootPattern-40689968 Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: e1012f856881485cb40d916131a5d763 Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-TestVolumeBootPattern-40689968-project Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.133431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '984863bd-b3a1-4546-8b36-720ba324d7ac'} Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:03:03.134253 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. Sep 11 01:03:03.136297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:03:03.137687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Returning 404 to user: Instance 984863bd-b3a1-4546-8b36-720ba324d7ac could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:03:03.137687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-33b16b6f-c9d6-4799-ae3f-0b8a9c8feb0e tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac" status: 404 len: 111 microversion: 2.1 time: 0.030182 Sep 11 01:03:03.137687 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 503/989] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:03 2022] GET /compute/v2.1/servers/984863bd-b3a1-4546-8b36-720ba324d7ac => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:03:04.428721 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-9633df1f-f09a-4d8e-9a67-2e1073a8d38b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] Calling method '.version_select at 0x7fc01addc5e0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:04.433539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR oslo.messaging._drivers.impl_rabbit [None req-9633df1f-f09a-4d8e-9a67-2e1073a8d38b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [95126ac8-fe1d-4805-b353-0d857b834e91] AMQP server on 10.222.0.27:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 11 01:03:05.440998 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO oslo.messaging._drivers.impl_rabbit [None req-9633df1f-f09a-4d8e-9a67-2e1073a8d38b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] [95126ac8-fe1d-4805-b353-0d857b834e91] Reconnected to AMQP server on 10.222.0.27:5672 via [amqp] client with port 37326. Sep 11 01:03:05.460073 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-9633df1f-f09a-4d8e-9a67-2e1073a8d38b tempest-TestVolumeBootPattern-40689968 tempest-TestVolumeBootPattern-40689968-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-975872804" status: 202 len: 0 microversion: 2.1 time: 1.035498 Sep 11 01:03:05.460640 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 487/990] 10.222.0.27 () {58 vars in 1260 bytes} [Sun Sep 11 01:03:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-975872804 => generated 0 bytes in 1037 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:03:16.928561 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:16.930739 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:16.933219 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 11 01:03:16.933368 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Sun Sep 11 01:03:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments (ip 10.222.0.27) !!! Sep 11 01:03:16.934390 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 11 01:03:16.934503 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Sun Sep 11 01:03:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments (ip 10.222.0.27) !!! Sep 11 01:03:16.935577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=5 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:16.936931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=5 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:16.944996 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:16.945485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:16.945917 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:16.946615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:16.947095 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:16.948110 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:16.980319 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8595305a-33f6-4463-b62f-08051a6d7088 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055830 Sep 11 01:03:16.980877 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 504/991] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 01:03:16 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 01:03:17.042169 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=13 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:17.042936 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:17.047300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:17.047803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:17.055484 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:17.055975 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:17.056418 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:17.057505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:17.057962 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:17.058387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:17.086977 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:03:17.162170 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-68db93ea-0894-4cf1-afcb-f05aa079091a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 204 len: 0 microversion: 2.1 time: 0.121469 Sep 11 01:03:17.162753 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 488/992] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:03:17 2022] DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 01:03:17.171394 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:17.172051 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:17.177853 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:17.178339 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:17.178774 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:17.179485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:17.179947 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:17.180387 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:17.309939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:17.317583 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-b1c6ecd2-7047-478e-b0a2-817c1c212819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1675 microversion: 2.1 time: 0.150105 Sep 11 01:03:17.318146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 505/993] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:17 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1675 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:18.341335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:18.342078 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:18.348471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:18.348964 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:18.349413 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:18.350110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:18.350636 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:18.351105 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:18.479392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:18.487234 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-79f17b53-0632-4d46-8b42-4d4164771c3b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1675 microversion: 2.1 time: 0.150276 Sep 11 01:03:18.487791 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 489/994] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:18 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1675 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:19.512229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:19.512772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:19.519120 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:19.519603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:19.520034 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:19.520720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:19.521169 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:19.521589 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:19.747014 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:19.754609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-ab7e6b65-97bc-403c-8895-4510333ac85b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1675 microversion: 2.1 time: 0.246925 Sep 11 01:03:19.755203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 506/995] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:19 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1675 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:20.577640 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-54da2555-8159-4eb9-92c5-ce82e3369c17 req-1807fd1f-0f76-4a69-b468-1975fd04e0cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0875c4c-2396-4199-bf90-c505cd4ab7f8", "name": "network-vif-deleted", "tag": "92f73122-c3f2-4b96-9d29-f12911010f02"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:20.588339 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-54da2555-8159-4eb9-92c5-ce82e3369c17 req-1807fd1f-0f76-4a69-b468-1975fd04e0cb service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:20.589107 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-54da2555-8159-4eb9-92c5-ce82e3369c17 req-1807fd1f-0f76-4a69-b468-1975fd04e0cb service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:20.589556 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-54da2555-8159-4eb9-92c5-ce82e3369c17 req-1807fd1f-0f76-4a69-b468-1975fd04e0cb service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:20.602082 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-54da2555-8159-4eb9-92c5-ce82e3369c17 req-1807fd1f-0f76-4a69-b468-1975fd04e0cb service nova] Creating event network-vif-deleted:92f73122-c3f2-4b96-9d29-f12911010f02 for instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 on hv-cinder-maste Sep 11 01:03:20.608172 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-54da2555-8159-4eb9-92c5-ce82e3369c17 req-1807fd1f-0f76-4a69-b468-1975fd04e0cb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.238168 Sep 11 01:03:20.608731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 490/996] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:03:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:03:20.779459 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:20.780083 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:20.787236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:20.787730 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:20.788166 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:20.789017 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:20.789461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:20.789884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:20.826437 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:20.827006 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:20.827392 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:20.827841 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.objects.instance [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lazy-loading 'fault' on Instance uuid b0875c4c-2396-4199-bf90-c505cd4ab7f8 {{(pid=126981) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:03:20.873169 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:20.888396 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cbfab54d-a8de-4dfe-96c5-73520454835b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 200 len: 1327 microversion: 2.1 time: 0.112870 Sep 11 01:03:20.897747 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 507/997] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:20 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 1327 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:21.913064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:21.913772 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:21.920974 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:21.921460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:21.921890 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:21.922574 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:21.923054 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:21.923474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:21.937164 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] HTTP exception thrown: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Request method failure captured: Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 HTTP/1.1 Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTMt5WekKs3f5Vlta4Get3rP45iOzrzR0TQwM89C1rpL1M-vJOsI636o_Ucmu_wqB-qjBWZq8dxZHmGgZ3XOc0i5MQ22nbuMxbZ_BM45GPA3rF5cuT5Xsb-8Txc7G_NLvHigZ0v0wQql8gZXcnxKP7UUN6Tipc_WSBhLWIiry4eWAJE Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:03:21.938775 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 58b3528fd06b4d438caa92fd88383ed5 Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:03:21.940537 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'b0875c4c-2396-4199-bf90-c505cd4ab7f8'} Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:03:21.941660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:03:21.943301 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:03:21.945289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Returning 404 to user: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:03:21.945289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d44607be-ac7e-4e69-a36f-e497e2d60d52 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 404 len: 111 microversion: 2.1 time: 0.031451 Sep 11 01:03:21.945289 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 491/998] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:21 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:03:21.949467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-1aaa4c02-201b-42d3-abed-a0350667a6af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '.version_select at 0x7fc01aaa2d30>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:21.974819 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-1aaa4c02-201b-42d3-abed-a0350667a6af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-993355735" status: 202 len: 0 microversion: 2.1 time: 0.029582 Sep 11 01:03:21.975384 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 508/999] 10.222.0.27 () {58 vars in 1218 bytes} [Sun Sep 11 01:03:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-993355735 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:03:22.586525 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-b755e9b0-67c5-46fa-a9aa-f0667ce7149c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: .version_select at 0x7fc01c3e5790>, body: {"keypair": {"name": "tempest-keypair-1467039418", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCSUWDs0MazVWFik+5SMbJ0UE3QYsmqdOZ7DQEl8d+GCHjvjDXr2nfYOI7Q2vzLdldkwxyR8RbjY/A67THf7HJUx5kpIcTU49P5OtRvOjEFtG4TSBhysIP/leHxEYZY9UA=="}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:22.603291 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-b755e9b0-67c5-46fa-a9aa-f0667ce7149c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for project 15810c36aaa44c5fbc87197f873a6260. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:03:22.607757 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-b755e9b0-67c5-46fa-a9aa-f0667ce7149c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for user 58b3528fd06b4d438caa92fd88383ed5 and project 15810c36aaa44c5fbc87197f873a6260. Resources: {'key_pairs'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:03:22.629092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-b755e9b0-67c5-46fa-a9aa-f0667ce7149c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.046455 Sep 11 01:03:22.629703 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 492/1000] 10.222.0.27 () {60 vars in 1159 bytes} [Sun Sep 11 01:03:22 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:03:23.885527 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1848032113", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "049b6c8b-1072-47b9-b740-d7f0f662cb7f"}], "security_groups": [{"name": "tempest-securitygroup--1711812906"}], "key_name": "tempest-keypair-1467039418", "user_data": "}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:23.989989 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] validate_networks() for [('049b6c8b-1072-47b9-b740-d7f0f662cb7f', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:03:24.363275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:03:24.363982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:03:24.364466 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:03:24.364881 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:03:24.365296 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:03:24.374633 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:24.375150 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:24.375589 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:24.376330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:24.376771 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:24.377196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:24.394364 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for project 15810c36aaa44c5fbc87197f873a6260. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:03:24.398420 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Getting quotas for user 58b3528fd06b4d438caa92fd88383ed5 and project 15810c36aaa44c5fbc87197f873a6260. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:03:24.409845 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:03:24.412160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:03:24.459959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-20c5715e-b6a2-4c69-94ac-eb32e1e2afeb tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.578941 Sep 11 01:03:24.460606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 509/1001] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:03:23 2022] POST /compute/v2.1/servers => generated 402 bytes in 580 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:03:24.472344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-db5c19c7-c529-4ca1-968a-571d394c65db tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:24.472965 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-db5c19c7-c529-4ca1-968a-571d394c65db tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:24.532370 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-db5c19c7-c529-4ca1-968a-571d394c65db tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:24.539284 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-db5c19c7-c529-4ca1-968a-571d394c65db tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1238 microversion: 2.1 time: 0.071305 Sep 11 01:03:24.539844 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 493/1002] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:24 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1238 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:25.563389 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:25.563985 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:25.571505 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:25.571990 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:25.573160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:25.573850 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:25.574304 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:25.574719 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:25.673636 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:25.685001 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f0f30613-75c4-47b6-bdd5-9a07720b396c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1298 microversion: 2.1 time: 0.125907 Sep 11 01:03:25.685584 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 510/1003] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:25 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1298 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:26.709023 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:26.709647 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:26.717033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:26.717802 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:26.718037 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:26.718778 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:26.719260 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:26.719659 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:26.951212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:26.958803 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d34a0ba0-8bc5-4d2a-9618-816af3f67515 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.253783 Sep 11 01:03:26.959396 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 494/1004] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:26 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:27.151501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-5d3735bb-503f-4f14-aa1d-eb715b00d307 req-ba019779-6bfa-4dd7-bda1-29ac4c713d5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50fb54a3-4f4f-4df5-8af8-53bb70c7666b", "tag": "fd9254ff-c49e-4ebc-8968-ba64656fa73d"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:27.162297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-5d3735bb-503f-4f14-aa1d-eb715b00d307 req-ba019779-6bfa-4dd7-bda1-29ac4c713d5e service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:27.162782 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-5d3735bb-503f-4f14-aa1d-eb715b00d307 req-ba019779-6bfa-4dd7-bda1-29ac4c713d5e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:27.163452 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-5d3735bb-503f-4f14-aa1d-eb715b00d307 req-ba019779-6bfa-4dd7-bda1-29ac4c713d5e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:27.175603 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-5d3735bb-503f-4f14-aa1d-eb715b00d307 req-ba019779-6bfa-4dd7-bda1-29ac4c713d5e service nova] Creating event network-changed:fd9254ff-c49e-4ebc-8968-ba64656fa73d for instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b on hv-cinder-maste Sep 11 01:03:27.180456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-5d3735bb-503f-4f14-aa1d-eb715b00d307 req-ba019779-6bfa-4dd7-bda1-29ac4c713d5e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033640 Sep 11 01:03:27.181061 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 511/1005] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:03:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:03:27.982578 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:27.983204 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:27.989737 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:27.990224 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:27.990660 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:27.991387 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:27.991845 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:27.992274 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:28.124880 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:28.132489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-15cd3869-895d-4931-bee7-961067b8e51b tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.154351 Sep 11 01:03:28.133048 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 495/1006] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:27 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:29.156309 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:29.157020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:29.163142 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:29.163645 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:29.164060 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:29.164743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:29.165196 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:29.165618 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:29.296026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:29.303609 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2d83cbbe-4375-4cb3-b258-a056bfd59bff tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.152318 Sep 11 01:03:29.304188 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 512/1007] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:29 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:30.326520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:30.327166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:30.333638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:30.334109 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:30.334549 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:30.335275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:30.335731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:30.336180 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:30.465697 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:30.473261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-2ae06b3b-3d2e-4632-853f-37c9a2abb45a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.151344 Sep 11 01:03:30.473829 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 496/1008] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:30 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:31.496501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:31.497137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:31.503554 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:31.504054 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:31.504523 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:31.505213 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:31.505664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:31.506082 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:31.637641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:31.645162 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-8684ec2f-ff96-4261-8ec7-cd0edc710906 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.153073 Sep 11 01:03:31.645720 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 513/1009] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:31 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:32.668184 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:32.668832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:32.675226 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:32.675723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:32.676161 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:32.676861 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:32.677316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:32.677740 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:32.792685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-e9705e31-8d65-4b9c-8d57-29755274b6b1 req-3629ba9d-6110-4ac3-9d22-e6cbee921155 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "50fb54a3-4f4f-4df5-8af8-53bb70c7666b", "name": "network-vif-plugged", "status": "completed", "tag": "fd9254ff-c49e-4ebc-8968-ba64656fa73d"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:32.802203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e9705e31-8d65-4b9c-8d57-29755274b6b1 req-3629ba9d-6110-4ac3-9d22-e6cbee921155 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:32.802685 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e9705e31-8d65-4b9c-8d57-29755274b6b1 req-3629ba9d-6110-4ac3-9d22-e6cbee921155 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:32.803162 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-e9705e31-8d65-4b9c-8d57-29755274b6b1 req-3629ba9d-6110-4ac3-9d22-e6cbee921155 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:32.814683 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-e9705e31-8d65-4b9c-8d57-29755274b6b1 req-3629ba9d-6110-4ac3-9d22-e6cbee921155 service nova] Creating event network-vif-plugged:fd9254ff-c49e-4ebc-8968-ba64656fa73d for instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b on hv-cinder-maste Sep 11 01:03:32.819255 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:32.819925 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-e9705e31-8d65-4b9c-8d57-29755274b6b1 req-3629ba9d-6110-4ac3-9d22-e6cbee921155 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101974 Sep 11 01:03:32.820515 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 514/1010] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:03:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:03:32.827832 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-c6739d47-7afc-47e8-913e-09774bbaf819 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.163585 Sep 11 01:03:32.828385 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 497/1011] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:32 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:33.851297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:33.852074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:33.858768 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:33.859281 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:33.859711 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:33.860391 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:33.860839 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:33.862229 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:33.998019 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:34.005614 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-14ff433c-b59e-4b9f-b096-36f8ca3b4c0a tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1364 microversion: 2.1 time: 0.159248 Sep 11 01:03:34.006173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 515/1012] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:33 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:03:35.029091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:35.029738 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:35.032506 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:35.038112 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:35.038607 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:35.039071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:35.039769 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:35.040245 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:35.040671 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:35.171637 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:03:35.179199 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-caeee1d5-cc65-49bf-9518-d55064c9c1c3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1550 microversion: 2.1 time: 0.154104 Sep 11 01:03:35.179761 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 498/1013] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:03:35 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1550 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:03:35.201057 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:03:35.201567 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:03:35.203485 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:35.204134 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:35.210441 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:35.210949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:35.211383 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:35.212069 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:35.212518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:35.212951 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:35.289734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4f7b58b7-6468-4501-b2b8-b558ea4ed726 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092504 Sep 11 01:03:35.290362 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 516/1014] 10.222.0.27 () {58 vars in 1275 bytes} [Sun Sep 11 01:03:35 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:03:36.195460 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-448564df-1417-4da4-bdad-b580eefbfad7 req-61ac0ace-0e6c-4145-980a-2a0e3a25d472 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50fb54a3-4f4f-4df5-8af8-53bb70c7666b", "tag": "fd9254ff-c49e-4ebc-8968-ba64656fa73d"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:03:36.206432 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-448564df-1417-4da4-bdad-b580eefbfad7 req-61ac0ace-0e6c-4145-980a-2a0e3a25d472 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:03:36.206944 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-448564df-1417-4da4-bdad-b580eefbfad7 req-61ac0ace-0e6c-4145-980a-2a0e3a25d472 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:03:36.207381 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-448564df-1417-4da4-bdad-b580eefbfad7 req-61ac0ace-0e6c-4145-980a-2a0e3a25d472 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:03:36.218662 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-448564df-1417-4da4-bdad-b580eefbfad7 req-61ac0ace-0e6c-4145-980a-2a0e3a25d472 service nova] Creating event network-changed:fd9254ff-c49e-4ebc-8968-ba64656fa73d for instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b on hv-cinder-maste Sep 11 01:03:36.223731 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-448564df-1417-4da4-bdad-b580eefbfad7 req-61ac0ace-0e6c-4145-980a-2a0e3a25d472 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032024 Sep 11 01:03:36.224754 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 499/1015] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:03:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:10.302778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:04:10.303648 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=60 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:04:10.308808 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:10.309300 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:10.316689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:10.317172 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:10.317598 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:10.318290 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:10.318734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:10.319179 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:10.345042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-84a2049b-c30f-44bc-abc3-5ff6d5142927 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/ips" status: 200 len: 146 microversion: 2.1 time: 0.043615 Sep 11 01:04:10.345601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 517/1016] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 01:04:10 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/ips => generated 146 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:11.678526 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "18dea625-e084-46f0-aa3b-bd78a4bb5d97"}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:04:11.681044 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:11.683412 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=40,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:04:11.684051 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:04:11.690081 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:11.690565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:11.691023 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:11.691707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:11.692159 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:11.692579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:11.952748 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5065cca7-1370-41f6-862c-43327f68e809 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.278333 Sep 11 01:04:11.953348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 500/1017] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 01:04:11 2022] POST /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments => generated 194 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:18.435955 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:18.438236 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:18.444641 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:18.445124 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:18.445564 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:18.446251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:18.446699 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:18.447149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:18.476343 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-9ae31e8a-27a8-40c1-b1a3-abc0f849cdc3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044373 Sep 11 01:04:18.476890 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 518/1018] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 01:04:18 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:18.489250 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:18.489871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:18.496091 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:18.497244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:18.497739 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:18.498426 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:18.498919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:18.499348 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:18.528288 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-352674c7-0293-436a-b03c-df31f9c6e80f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/18dea625-e084-46f0-aa3b-bd78a4bb5d97" status: 200 len: 194 microversion: 2.1 time: 0.043746 Sep 11 01:04:18.528784 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 501/1019] 10.222.0.27 () {58 vars in 1413 bytes} [Sun Sep 11 01:04:18 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/18dea625-e084-46f0-aa3b-bd78a4bb5d97 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:19.851981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "972a7c6b-3648-41e2-9580-3c7805e7bda2"}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:04:19.854282 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:19.860659 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:19.861141 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:19.861574 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:19.862260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:19.862708 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:19.863149 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:20.125408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-28c4e3f7-4ad0-483a-8fe7-d1578237c8af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "POST /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.276957 Sep 11 01:04:20.126026 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 519/1020] 10.222.0.27 () {60 vars in 1323 bytes} [Sun Sep 11 01:04:19 2022] POST /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments => generated 194 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:27.662707 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:27.665003 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:27.671867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:27.672352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:27.672787 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:27.673476 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:27.673931 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:27.674352 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:27.704567 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-1358fd0a-6cc9-43a2-b427-f890cb401594 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.045758 Sep 11 01:04:27.705103 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 502/1021] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 01:04:27 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments => generated 371 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:27.850533 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:27.851114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:27.857968 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:27.858454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:27.858931 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:27.859629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:27.860086 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:27.860509 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:27.888803 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-f49c3a5e-6761-454c-a72e-c12df5d20bf0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/18dea625-e084-46f0-aa3b-bd78a4bb5d97" status: 200 len: 194 microversion: 2.1 time: 0.042641 Sep 11 01:04:27.889761 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 520/1022] 10.222.0.27 () {58 vars in 1413 bytes} [Sun Sep 11 01:04:27 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/18dea625-e084-46f0-aa3b-bd78a4bb5d97 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:27.900912 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:27.901421 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:27.908669 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:27.909163 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:27.909604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:27.910297 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:27.910758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:27.911217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:27.991900 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lazy-loading 'services' on Instance uuid 50fb54a3-4f4f-4df5-8af8-53bb70c7666b {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:04:28.063995 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-50d256f8-db6b-4760-a9cb-6efac9875505 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/18dea625-e084-46f0-aa3b-bd78a4bb5d97" status: 202 len: 0 microversion: 2.1 time: 0.167165 Sep 11 01:04:28.064579 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 503/1023] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 01:04:27 2022] DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/18dea625-e084-46f0-aa3b-bd78a4bb5d97 => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:04:29.192206 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:29.192725 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:29.199165 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:29.199650 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:29.200112 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:29.200800 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:29.201251 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:29.201676 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:29.230709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-c59a5492-e602-47d8-ac5d-314b4273a01f tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/972a7c6b-3648-41e2-9580-3c7805e7bda2" status: 200 len: 194 microversion: 2.1 time: 0.042392 Sep 11 01:04:29.232015 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 521/1024] 10.222.0.27 () {58 vars in 1413 bytes} [Sun Sep 11 01:04:29 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/972a7c6b-3648-41e2-9580-3c7805e7bda2 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:29.243834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:29.244431 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:29.251183 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:29.251675 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:29.252115 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:29.252807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:29.253266 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:29.253695 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:29.334536 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lazy-loading 'services' on Instance uuid 50fb54a3-4f4f-4df5-8af8-53bb70c7666b {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:04:29.405442 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-eb3263c7-7523-4c48-a36b-7c2ca872f9f5 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/972a7c6b-3648-41e2-9580-3c7805e7bda2" status: 202 len: 0 microversion: 2.1 time: 0.165768 Sep 11 01:04:29.406013 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 504/1025] 10.222.0.27 () {58 vars in 1416 bytes} [Sun Sep 11 01:04:29 2022] DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments/972a7c6b-3648-41e2-9580-3c7805e7bda2 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:04:30.580580 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:30.582884 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:30.589260 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.589704 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.590132 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.590829 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.591280 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.591695 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.622972 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-35be7db5-5ec7-46a4-ac5f-cf32a1c16517 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046255 Sep 11 01:04:30.623519 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 522/1026] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 01:04:30 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 01:04:30.738305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:30.740561 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:30.747225 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.747716 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.748156 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.748839 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.749293 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.749725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.778475 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-11e933ff-9f6b-4f86-a11b-ceef8e78f9a0 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044903 Sep 11 01:04:30.779028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 505/1027] 10.222.0.27 () {58 vars in 1302 bytes} [Sun Sep 11 01:04:30 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 11 01:04:30.842233 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:30.842664 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:30.848501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.848978 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.849415 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.850091 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.850536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.850981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.876431 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Going to try to terminate instance {{(pid=126981) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:04:30.952528 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4e818a4a-b1b6-4248-8b6d-4f80be5538a3 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 204 len: 0 microversion: 2.1 time: 0.114080 Sep 11 01:04:30.953144 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 523/1028] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:04:30 2022] DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 01:04:30.961667 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:30.962316 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:30.968925 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.969422 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.969867 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:30.970570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:30.971066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:30.971500 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:31.103805 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:04:31.113137 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f7e6016a-e048-4a61-a683-f086a74777af tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1675 microversion: 2.1 time: 0.155363 Sep 11 01:04:31.113691 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 506/1029] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:04:30 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1675 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:04:32.137949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:32.138675 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:32.145059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:32.145544 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:32.145981 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:32.146665 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:32.147152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:32.147578 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:32.274077 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:04:32.281770 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-25fdb608-940c-4875-82b5-61b64ec28db7 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1675 microversion: 2.1 time: 0.148656 Sep 11 01:04:32.282326 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 524/1030] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:04:32 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1675 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:04:33.307691 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:33.308513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:33.315743 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:33.316309 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:33.316837 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:33.317623 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:33.318168 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:33.318663 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:33.412443 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58b3528fd06b4d438caa92fd88383ed5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15810c36aaa44c5fbc87197f873a6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:04:33.421764 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d0653a50-1dcb-481b-9b41-397d0a49ec0c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 200 len: 1607 microversion: 2.1 time: 0.119868 Sep 11 01:04:33.435919 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 507/1031] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:04:33 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 1607 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:04:33.602263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-1eb59f6d-9cf2-43a1-a0e7-49236d5657ba req-388921e1-bcec-4bc7-a599-82b64be85404 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "50fb54a3-4f4f-4df5-8af8-53bb70c7666b", "name": "network-vif-deleted", "tag": "fd9254ff-c49e-4ebc-8968-ba64656fa73d"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:04:33.612883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-1eb59f6d-9cf2-43a1-a0e7-49236d5657ba req-388921e1-bcec-4bc7-a599-82b64be85404 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:33.613345 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-1eb59f6d-9cf2-43a1-a0e7-49236d5657ba req-388921e1-bcec-4bc7-a599-82b64be85404 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:33.613760 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-1eb59f6d-9cf2-43a1-a0e7-49236d5657ba req-388921e1-bcec-4bc7-a599-82b64be85404 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:33.626063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-1eb59f6d-9cf2-43a1-a0e7-49236d5657ba req-388921e1-bcec-4bc7-a599-82b64be85404 service nova] Creating event network-vif-deleted:fd9254ff-c49e-4ebc-8968-ba64656fa73d for instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b on hv-cinder-maste Sep 11 01:04:33.632891 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-1eb59f6d-9cf2-43a1-a0e7-49236d5657ba req-388921e1-bcec-4bc7-a599-82b64be85404 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034507 Sep 11 01:04:33.633483 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 525/1032] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:04:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:34.447628 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:34.448344 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:34.456066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:34.456564 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:34.457005 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:34.457725 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:34.458208 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:34.458638 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:34.473755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] HTTP exception thrown: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Request method failure captured: Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b HTTP/1.1 Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTMt5WekKs3f5Vlta4Get3rP45iOzrzR0TQwM89C1rpL1M-vJOsI636o_Ucmu_wqB-qjBWZq8dxZHmGgZ3XOc0i5MQ22nbuMxbZ_BM45GPA3rF5cuT5Xsb-8Txc7G_NLvHigZ0v0wQql8gZXcnxKP7UUN6Tipc_WSBhLWIiry4eWAJE Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:04:34.475474 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 58b3528fd06b4d438caa92fd88383ed5 Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:34.477222 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '50fb54a3-4f4f-4df5-8af8-53bb70c7666b'} Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:04:34.478217 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:34.479984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:34.481644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Returning 404 to user: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:04:34.481644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3d25c039-3537-40fa-b0c4-0925d290e215 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 404 len: 111 microversion: 2.1 time: 0.033299 Sep 11 01:04:34.481644 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 508/1033] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:04:34 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:04:34.488283 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-54f89637-7fda-4560-bce9-a3928b273d3c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '.version_select at 0x7fc01aaa7f70>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:34.507419 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-54f89637-7fda-4560-bce9-a3928b273d3c tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1467039418" status: 202 len: 0 microversion: 2.1 time: 0.023530 Sep 11 01:04:34.508223 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 526/1034] 10.222.0.27 () {58 vars in 1221 bytes} [Sun Sep 11 01:04:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1467039418 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:04:37.380100 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:37.380605 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:37.386913 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:37.387451 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:37.387887 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:37.388572 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:37.389029 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:37.389456 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:37.402550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] HTTP exception thrown: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Request method failure captured: Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b HTTP/1.1 Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTMt5WekKs3f5Vlta4Get3rP45iOzrzR0TQwM89C1rpL1M-vJOsI636o_Ucmu_wqB-qjBWZq8dxZHmGgZ3XOc0i5MQ22nbuMxbZ_BM45GPA3rF5cuT5Xsb-8Txc7G_NLvHigZ0v0wQql8gZXcnxKP7UUN6Tipc_WSBhLWIiry4eWAJE Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:04:37.404817 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 58b3528fd06b4d438caa92fd88383ed5 Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.406471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '50fb54a3-4f4f-4df5-8af8-53bb70c7666b'} Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:04:37.407513 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.409064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.410570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Returning 404 to user: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:04:37.410570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-95566830-9bdd-43b2-a095-87e87dd9e100 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 404 len: 111 microversion: 2.1 time: 0.029964 Sep 11 01:04:37.410570 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 509/1035] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:04:37 2022] DELETE /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:04:37.414838 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:37.415572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: 50fb54a3-4f4f-4df5-8af8-53bb70c7666b] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:37.423352 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:37.423831 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:37.424262 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:37.424949 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:37.425408 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:37.425851 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:37.438757 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] HTTP exception thrown: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Request method failure captured: Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b HTTP/1.1 Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTMt5WekKs3f5Vlta4Get3rP45iOzrzR0TQwM89C1rpL1M-vJOsI636o_Ucmu_wqB-qjBWZq8dxZHmGgZ3XOc0i5MQ22nbuMxbZ_BM45GPA3rF5cuT5Xsb-8Txc7G_NLvHigZ0v0wQql8gZXcnxKP7UUN6Tipc_WSBhLWIiry4eWAJE Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:04:37.440401 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 58b3528fd06b4d438caa92fd88383ed5 Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.442063 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '50fb54a3-4f4f-4df5-8af8-53bb70c7666b'} Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:04:37.442941 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. Sep 11 01:04:37.444265 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:37.445518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Returning 404 to user: Instance 50fb54a3-4f4f-4df5-8af8-53bb70c7666b could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:04:37.445518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-027ad496-e8aa-4dcf-ae93-15f94f9d6ac6 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b" status: 404 len: 111 microversion: 2.1 time: 0.030818 Sep 11 01:04:37.445518 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 527/1036] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:04:37 2022] GET /compute/v2.1/servers/50fb54a3-4f4f-4df5-8af8-53bb70c7666b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:04:38.579261 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:38.579774 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:38.586135 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:38.586616 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:38.587076 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:38.587760 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:38.588212 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:38.588630 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:38.603925 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] HTTP exception thrown: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Request method failure captured: Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 HTTP/1.1 Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTMt5WekKs3f5Vlta4Get3rP45iOzrzR0TQwM89C1rpL1M-vJOsI636o_Ucmu_wqB-qjBWZq8dxZHmGgZ3XOc0i5MQ22nbuMxbZ_BM45GPA3rF5cuT5Xsb-8Txc7G_NLvHigZ0v0wQql8gZXcnxKP7UUN6Tipc_WSBhLWIiry4eWAJE Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:04:38.605492 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 58b3528fd06b4d438caa92fd88383ed5 Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: > Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.607914 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': 'b0875c4c-2396-4199-bf90-c505cd4ab7f8'} Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:04:38.608885 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.610515 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.612075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Returning 404 to user: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:04:38.612075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4ea45d4c-b697-4e06-a3cd-c3178a14ff9d tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 404 len: 111 microversion: 2.1 time: 0.032929 Sep 11 01:04:38.612075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 510/1037] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:04:38 2022] DELETE /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:04:38.616629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:38.617297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] [instance: b0875c4c-2396-4199-bf90-c505cd4ab7f8] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:04:38.624520 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:38.625004 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:38.625435 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:38.626125 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:04:38.626577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:04:38.627061 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:04:38.640062 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] HTTP exception thrown: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Request method failure captured: Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 HTTP/1.1 Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTMt5WekKs3f5Vlta4Get3rP45iOzrzR0TQwM89C1rpL1M-vJOsI636o_Ucmu_wqB-qjBWZq8dxZHmGgZ3XOc0i5MQ22nbuMxbZ_BM45GPA3rF5cuT5Xsb-8Txc7G_NLvHigZ0v0wQql8gZXcnxKP7UUN6Tipc_WSBhLWIiry4eWAJE Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:04:38.641600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_15810c36aaa44c5fbc87197f873a6260"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 15810c36aaa44c5fbc87197f873a6260 Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1403860877 Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: 58b3528fd06b4d438caa92fd88383ed5 Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-AttachVolumeTestJSON-1403860877-project Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.643272 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'b0875c4c-2396-4199-bf90-c505cd4ab7f8'} Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:04:38.644203 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. Sep 11 01:04:38.645732 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:04:38.647154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] Returning 404 to user: Instance b0875c4c-2396-4199-bf90-c505cd4ab7f8 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:04:38.647154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-e8ae246c-ebed-499d-8e47-8610711e30d9 tempest-AttachVolumeTestJSON-1403860877 tempest-AttachVolumeTestJSON-1403860877-project] 10.222.0.27 "GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8" status: 404 len: 111 microversion: 2.1 time: 0.030260 Sep 11 01:04:38.647154 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 528/1038] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:04:38 2022] GET /compute/v2.1/servers/b0875c4c-2396-4199-bf90-c505cd4ab7f8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:04:43.035196 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-58382b97-087c-40fb-85df-267b741840a6 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Action: 'create', calling method: .version_select at 0x7fc01aebf940>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1446711053", "metadata": {"Type": "work"}}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:04:43.319746 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-58382b97-087c-40fb-85df-267b741840a6 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.397899 Sep 11 01:04:43.320758 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 511/1039] 10.222.0.27 () {60 vars in 1156 bytes} [Sun Sep 11 01:04:42 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 399 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 11 01:04:43.334991 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-cd7acfa7-946f-472e-922c-c18f2f37c7f1 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Calling method '.version_select at 0x7fc01ac7e700>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:43.388346 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-cd7acfa7-946f-472e-922c-c18f2f37c7f1 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e" status: 200 len: 353 microversion: 2.1 time: 0.057626 Sep 11 01:04:43.388910 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 529/1040] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 01:04:43 2022] GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e => generated 353 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:44.403114 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-ef0169df-b079-4f4c-84bd-fe6f42c64223 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Calling method '.version_select at 0x7fc01ae70550>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:44.452002 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-ef0169df-b079-4f4c-84bd-fe6f42c64223 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e" status: 200 len: 354 microversion: 2.1 time: 0.053227 Sep 11 01:04:44.452597 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 512/1041] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 01:04:44 2022] GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e => generated 354 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:44.465778 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-6629595d-9614-45f5-8532-4f7e8fc2be35 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Calling method '.version_select at 0x7fc01ac813a0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:44.511161 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-6629595d-9614-45f5-8532-4f7e8fc2be35 tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e" status: 200 len: 354 microversion: 2.1 time: 0.049606 Sep 11 01:04:44.511723 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 530/1042] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 01:04:44 2022] GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e => generated 354 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:44.529858 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-62e50d4c-af5b-41af-b59f-8c84b4017acd tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Calling method '.version_select at 0x7fc01a92b700>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:44.590830 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-62e50d4c-af5b-41af-b59f-8c84b4017acd tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e" status: 202 len: 0 microversion: 2.1 time: 0.064633 Sep 11 01:04:44.591411 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 513/1043] 10.222.0.27 () {58 vars in 1248 bytes} [Sun Sep 11 01:04:44 2022] DELETE /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e => generated 0 bytes in 66 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 11 01:04:44.599297 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-7ce6f4a7-dc79-417e-b84b-96606343c99e tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Calling method '.version_select at 0x7fc01ac81af0>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:44.648858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-7ce6f4a7-dc79-417e-b84b-96606343c99e tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e" status: 200 len: 353 microversion: 2.1 time: 0.053310 Sep 11 01:04:44.649454 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 531/1044] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 01:04:44 2022] GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e => generated 353 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:04:45.663904 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5fbacfa4-e0ea-427e-808c-8f09e358853f tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Calling method '.version_select at 0x7fc01ba95dc0>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:04:45.692118 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.wsgi [None req-5fbacfa4-e0ea-427e-808c-8f09e358853f tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] HTTP exception thrown: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5fbacfa4-e0ea-427e-808c-8f09e358853f tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Request method failure captured: Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: request: GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e HTTP/1.1 Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept: application/json Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Accept-Encoding: identity Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Connection: close Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Length: 0 Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Content-Type: application/json Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Host: 10.222.0.27 Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: Openstack-System-Scope: None Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: User-Agent: python-urllib3/1.26.12 Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Auth-Token: gAAAAABjHTQqQNZxLxvUehmNHikFqOTwThw2gmFQP7eVEvl8bA8Pj5toIA7Rd8uO8-iyaCDF55HfEVO994211VuRzhYkYPw6HOOxY4E-Qwh2QgMg73JHsxEeIo-tARgwD8NHqZTyv2GyQe15mNbSxy-nsd-5WOo0ZvAM8sYjIwQCMdbKY7LrgYI Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Id: None Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Domain-Name: None Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Identity-Status: Confirmed Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Is-Admin-Project: True Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Id: default Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Domain-Name: Default Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Id: 560fe0b4cf484ccdb4bb441eb14b632f Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Project-Name: tempest-VolumesGetTestJSON-2111034391 Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Role: member,reader Sep 11 01:04:45.694065 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Roles: member,reader Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/560fe0b4cf484ccdb4bb441eb14b632f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/560fe0b4cf484ccdb4bb441eb14b632f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/560fe0b4cf484ccdb4bb441eb14b632f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_560fe0b4cf484ccdb4bb441eb14b632f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant: tempest-VolumesGetTestJSON-2111034391 Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Id: 560fe0b4cf484ccdb4bb441eb14b632f Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-Tenant-Name: tempest-VolumesGetTestJSON-2111034391 Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User: tempest-VolumesGetTestJSON-2111034391-project Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Id: default Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Domain-Name: Default Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Id: 0b22fdbb52764cb1b0512a5e157720ae Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: X-User-Name: tempest-VolumesGetTestJSON-2111034391-project Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: method: Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: exception: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: action_args: {'id': '517d1d02-f095-419b-b839-1a14378e3f2e'} Sep 11 01:04:45.695723 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. (HTTP 404) (Request-ID: req-ecb5dbeb-1f2a-4053-83c9-5aa44837c39d) Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:45.696755 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 11 01:04:45.698244 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 11 01:04:45.700645 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: ERROR nova.api.openstack.wsgi Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-5fbacfa4-e0ea-427e-808c-8f09e358853f tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] Returning 404 to user: Volume 517d1d02-f095-419b-b839-1a14378e3f2e could not be found. {{(pid=126979) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-5fbacfa4-e0ea-427e-808c-8f09e358853f tempest-VolumesGetTestJSON-2111034391 tempest-VolumesGetTestJSON-2111034391-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e" status: 404 len: 109 microversion: 2.1 time: 0.035979 Sep 11 01:04:45.702120 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 514/1045] 10.222.0.27 () {58 vars in 1245 bytes} [Sun Sep 11 01:04:45 2022] GET /compute/v2.1/os-volumes/517d1d02-f095-419b-b839-1a14378e3f2e => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:05:06.422498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:06.423275 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:06.427783 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1852536752", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "networks": [{"uuid": "27b0e7b0-ea93-4967-b24f-1fd576c9b236"}]}} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:06.483245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.network.neutron [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] validate_networks() for [('27b0e7b0-ea93-4967-b24f-1fd576c9b236', None, None, None, None, None)] {{(pid=126981) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:05:06.877045 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Flavor limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:05:06.877491 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Image limits 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:05:06.877965 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Flavor pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:05:06.878403 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Image pref 0:0:0 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:05:06.878840 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.virt.hardware [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126981) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:05:06.887568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:06.888737 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:06.889177 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:06.889907 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:06.890359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:06.890781 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:06.907170 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Getting quotas for project 74dc2b85295647929cd675c10ea882c7. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:05:06.911358 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.quota [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Getting quotas for user f63f7c17ba29451b8ba9599636c17e67 and project 74dc2b85295647929cd675c10ea882c7. Resources: {'cores', 'ram', 'instances'} {{(pid=126981) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:05:06.922172 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Going to run 1 instances... {{(pid=126981) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:05:06.924462 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126981) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:05:06.971876 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-911541b5-4520-4105-8d80-c2526177ed34 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.550618 Sep 11 01:05:06.972530 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 532/1046] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:05:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 552 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:05:06.981857 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_api:SELECT=22 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:06.983049 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:06.988138 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-fac9c49c-ea3c-4cb0-acd4-2a2e9b175341 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:06.988759 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-fac9c49c-ea3c-4cb0-acd4-2a2e9b175341 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:07.045471 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-fac9c49c-ea3c-4cb0-acd4-2a2e9b175341 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:07.052331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-fac9c49c-ea3c-4cb0-acd4-2a2e9b175341 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1214 microversion: 2.1 time: 0.072577 Sep 11 01:05:07.052883 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 515/1047] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:06 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1214 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:08.080750 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:08.080750 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:08.086348 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:08.086883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:08.087331 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:08.088050 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:08.088509 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:08.088937 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:08.201678 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:08.210047 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-d6c2bf52-d615-4013-a426-9b92b9f5186c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1284 microversion: 2.1 time: 0.137858 Sep 11 01:05:08.210911 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 533/1048] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:08 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1284 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:09.233892 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:09.234553 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:09.240911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:09.241404 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:09.241846 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:09.242547 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:09.243028 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:09.243464 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:09.397467 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-0d4f7e34-bcbb-4769-ae2b-eb7096e224a5 req-b9e0e730-f18c-4711-82e3-189cc9cad299 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8fe76d9-c41d-4dc5-939c-e75a204ad726", "tag": "f266a939-e0f7-4249-b8d1-d5893f78ad7a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:09.409571 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-0d4f7e34-bcbb-4769-ae2b-eb7096e224a5 req-b9e0e730-f18c-4711-82e3-189cc9cad299 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:09.410059 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-0d4f7e34-bcbb-4769-ae2b-eb7096e224a5 req-b9e0e730-f18c-4711-82e3-189cc9cad299 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:09.410488 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-0d4f7e34-bcbb-4769-ae2b-eb7096e224a5 req-b9e0e730-f18c-4711-82e3-189cc9cad299 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:09.422506 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-0d4f7e34-bcbb-4769-ae2b-eb7096e224a5 req-b9e0e730-f18c-4711-82e3-189cc9cad299 service nova] Creating event network-changed:f266a939-e0f7-4249-b8d1-d5893f78ad7a for instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 on hv-cinder-maste Sep 11 01:05:09.427461 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-0d4f7e34-bcbb-4769-ae2b-eb7096e224a5 req-b9e0e730-f18c-4711-82e3-189cc9cad299 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034968 Sep 11 01:05:09.428398 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 534/1049] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:05:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:05:09.459409 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:09.467361 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-984e8352-19cc-4157-a200-b9d39947c045 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1314 microversion: 2.1 time: 0.237450 Sep 11 01:05:09.467943 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 516/1050] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:09 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1314 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:10.492997 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:10.493692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:10.501456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:10.501933 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:10.502359 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:10.503075 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:10.503526 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:10.503939 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:10.732897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:10.740464 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-01e0b2a6-49eb-4cdd-9f6e-024542643832 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1314 microversion: 2.1 time: 0.252091 Sep 11 01:05:10.741126 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 535/1051] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:10 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1314 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:11.765852 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:11.766494 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:11.774898 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:11.775581 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:11.776020 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:11.776713 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:11.777166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:11.777614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:11.909349 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:11.916894 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-7a928c4d-533e-4edc-b7a1-4797078dbff4 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1314 microversion: 2.1 time: 0.156522 Sep 11 01:05:11.917450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 517/1052] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:11 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:12.940018 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:12.940631 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:12.947123 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:12.947610 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:12.948044 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:12.948736 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:12.949189 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:12.949615 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:13.082833 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:13.090511 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-95266c08-f70a-4c65-9f89-5e7d0c7d12e3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1314 microversion: 2.1 time: 0.154428 Sep 11 01:05:13.091855 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 536/1053] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:12 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 11 01:05:14.113392 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:14.114015 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:14.120782 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:14.121300 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:14.121741 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:14.122990 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:14.123454 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:14.123882 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:14.252116 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:14.259739 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-a4151773-a121-4ccf-9668-fee4190fabb7 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1314 microversion: 2.1 time: 0.150190 Sep 11 01:05:14.260306 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 518/1054] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:14 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:15.187858 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-7a7402f0-c904-4cc3-a863-cf493619bf09 req-e8bb3ec6-2edf-44fa-bb41-cf15772cabe7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8fe76d9-c41d-4dc5-939c-e75a204ad726", "name": "network-vif-plugged", "status": "completed", "tag": "f266a939-e0f7-4249-b8d1-d5893f78ad7a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:15.198587 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7a7402f0-c904-4cc3-a863-cf493619bf09 req-e8bb3ec6-2edf-44fa-bb41-cf15772cabe7 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:15.199105 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7a7402f0-c904-4cc3-a863-cf493619bf09 req-e8bb3ec6-2edf-44fa-bb41-cf15772cabe7 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:15.199542 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-7a7402f0-c904-4cc3-a863-cf493619bf09 req-e8bb3ec6-2edf-44fa-bb41-cf15772cabe7 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:15.210591 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-7a7402f0-c904-4cc3-a863-cf493619bf09 req-e8bb3ec6-2edf-44fa-bb41-cf15772cabe7 service nova] Creating event network-vif-plugged:f266a939-e0f7-4249-b8d1-d5893f78ad7a for instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 on hv-cinder-maste Sep 11 01:05:15.215536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-7a7402f0-c904-4cc3-a863-cf493619bf09 req-e8bb3ec6-2edf-44fa-bb41-cf15772cabe7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031352 Sep 11 01:05:15.216654 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 537/1055] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:05:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 11 01:05:15.285072 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:15.285691 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:15.292092 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:15.292604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:15.293042 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:15.294380 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:15.295110 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:15.295625 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:15.472930 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:15.480548 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-6aa37cb6-a8be-4bc0-8618-86c014de6e75 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1314 microversion: 2.1 time: 0.200801 Sep 11 01:05:15.481108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 519/1056] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:15 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1314 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:16.505450 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:16.506146 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:16.512266 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:16.512813 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:16.513259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:16.514160 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:16.514619 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:16.515074 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:16.645055 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:16.652728 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-50713d5a-8b8d-430b-b193-70fb115bd27b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1499 microversion: 2.1 time: 0.151947 Sep 11 01:05:16.653796 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 538/1057] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:16 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1499 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:17.127305 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:17.127884 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:17.130200 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:17.135419 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:17.135939 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:17.136377 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:17.137066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:17.137520 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:17.138550 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:17.163489 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:05:17.235613 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-df7f3d55-9c60-4d07-a85d-20874a2b997e tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "DELETE /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 204 len: 0 microversion: 2.1 time: 0.111921 Sep 11 01:05:17.236303 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 520/1058] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:05:17 2022] DELETE /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 01:05:17.245565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:17.246199 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:17.248959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:17.249647 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=6 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:17.255668 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:17.256133 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:17.256565 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:17.257245 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:17.257690 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:17.258106 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:17.390176 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:17.398020 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-3a565772-057f-4140-98fa-591194ff2e16 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1505 microversion: 2.1 time: 0.157143 Sep 11 01:05:17.398572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 539/1059] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:17 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1505 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:18.423063 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:18.423770 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:18.429653 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:18.430131 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:18.430565 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:18.431291 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:18.431744 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:18.432166 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:18.566378 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:18.574093 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-eee13445-6448-4301-87b1-7967a69256ab tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1505 microversion: 2.1 time: 0.156126 Sep 11 01:05:18.574652 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 521/1060] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:18 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1505 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:19.227126 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-be0de2ee-e810-434f-86eb-101f4dd0efdb req-f4a950f2-d1a2-48ee-9b17-6a9952df8c36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8fe76d9-c41d-4dc5-939c-e75a204ad726", "name": "network-vif-deleted", "tag": "f266a939-e0f7-4249-b8d1-d5893f78ad7a"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:19.237130 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-be0de2ee-e810-434f-86eb-101f4dd0efdb req-f4a950f2-d1a2-48ee-9b17-6a9952df8c36 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:19.237606 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-be0de2ee-e810-434f-86eb-101f4dd0efdb req-f4a950f2-d1a2-48ee-9b17-6a9952df8c36 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:19.238029 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-be0de2ee-e810-434f-86eb-101f4dd0efdb req-f4a950f2-d1a2-48ee-9b17-6a9952df8c36 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:19.248883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-be0de2ee-e810-434f-86eb-101f4dd0efdb req-f4a950f2-d1a2-48ee-9b17-6a9952df8c36 service nova] Creating event network-vif-deleted:f266a939-e0f7-4249-b8d1-d5893f78ad7a for instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 on hv-cinder-maste Sep 11 01:05:19.253923 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-be0de2ee-e810-434f-86eb-101f4dd0efdb req-f4a950f2-d1a2-48ee-9b17-6a9952df8c36 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030582 Sep 11 01:05:19.254772 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 540/1061] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:05:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:05:19.597331 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:19.597973 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:19.604857 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:19.605430 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:19.605961 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:19.606752 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:19.607321 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:19.607789 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:19.639201 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:19.639795 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:19.640794 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:19.641439 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lazy-loading 'fault' on Instance uuid b8fe76d9-c41d-4dc5-939c-e75a204ad726 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:05:19.687275 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:19.695682 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d0957e35-0dd0-4b26-b729-a9fbd2a7ddef tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 200 len: 1304 microversion: 2.1 time: 0.102171 Sep 11 01:05:19.696356 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 522/1062] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:19 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 1304 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:20.721919 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:20.722502 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: b8fe76d9-c41d-4dc5-939c-e75a204ad726] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:20.728577 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:20.729054 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:20.729480 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:20.730153 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:20.730600 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:20.731038 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:20.746601 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] HTTP exception thrown: Instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 could not be found. Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Request method failure captured: Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 HTTP/1.1 Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTRANQYY5al30RkAIWxXgCCXgmc46awUuThjf1OBsfMox6Dv_YZzbDMbWCw5n7n_7JNLHMM1uulGXR2L935P5LiIYkB9S7s0weGSK88PbEh15KOoTVWlaBp8t2HGwm3BA4O4SngPn5K-Ypfh6ZjbtfNi7ChT5IDb6F3d1I0dghtM3zc Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 74dc2b85295647929cd675c10ea882c7 Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesActionsTest-1078823862 Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:05:20.749263 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/74dc2b85295647929cd675c10ea882c7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/74dc2b85295647929cd675c10ea882c7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/74dc2b85295647929cd675c10ea882c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_74dc2b85295647929cd675c10ea882c7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesActionsTest-1078823862 Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 74dc2b85295647929cd675c10ea882c7 Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesActionsTest-1078823862 Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesActionsTest-1078823862-project Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: f63f7c17ba29451b8ba9599636c17e67 Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesActionsTest-1078823862-project Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 could not be found. Sep 11 01:05:20.751173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': 'b8fe76d9-c41d-4dc5-939c-e75a204ad726'} Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 could not be found. Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:05:20.752261 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 could not be found. Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 could not be found. Sep 11 01:05:20.754152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:20.755573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Returning 404 to user: Instance b8fe76d9-c41d-4dc5-939c-e75a204ad726 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:05:20.755573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-bfaaf971-5b1f-443f-ad2d-2e66e79412c6 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726" status: 404 len: 111 microversion: 2.1 time: 0.034499 Sep 11 01:05:20.755573 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 541/1063] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:20 2022] GET /compute/v2.1/servers/b8fe76d9-c41d-4dc5-939c-e75a204ad726 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 11 01:05:20.799271 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1273724898", "imageRef": "b9f199b2-49ec-41ee-ad99-4fb9c69dbe72", "flavorRef": "452", "networks": [{"uuid": "27b0e7b0-ea93-4967-b24f-1fd576c9b236"}]}} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:20.851886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.network.neutron [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] validate_networks() for [('27b0e7b0-ea93-4967-b24f-1fd576c9b236', None, None, None, None, None)] {{(pid=126979) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 11 01:05:21.248589 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Flavor limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 11 01:05:21.249024 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Image limits 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 11 01:05:21.249507 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Flavor pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 11 01:05:21.249959 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Image pref 0:0:0 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 11 01:05:21.250373 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.virt.hardware [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=126979) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 11 01:05:21.259053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:21.259539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:21.259984 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:21.260720 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:21.261179 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:21.261614 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:21.278260 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Getting quotas for project 74dc2b85295647929cd675c10ea882c7. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 11 01:05:21.282359 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.quota [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Getting quotas for user f63f7c17ba29451b8ba9599636c17e67 and project 74dc2b85295647929cd675c10ea882c7. Resources: {'cores', 'ram', 'instances'} {{(pid=126979) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 11 01:05:21.293643 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Going to run 1 instances... {{(pid=126979) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 11 01:05:21.296670 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b9f199b2-49ec-41ee-ad99-4fb9c69dbe72',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=126979) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 11 01:05:21.343855 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0c4e12f9-7d54-423a-98c7-19d69e8e6ea3 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.548386 Sep 11 01:05:21.350505 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 523/1064] 10.222.0.27 () {60 vars in 1147 bytes} [Sun Sep 11 01:05:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 556 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 11 01:05:21.355494 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-4168df07-1f95-4670-a9b0-aa649b50bdfe tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:21.356114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-4168df07-1f95-4670-a9b0-aa649b50bdfe tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:21.414443 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-4168df07-1f95-4670-a9b0-aa649b50bdfe tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:21.421550 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-4168df07-1f95-4670-a9b0-aa649b50bdfe tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1214 microversion: 2.1 time: 0.069790 Sep 11 01:05:21.422143 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 542/1065] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:21 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1214 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:22.444807 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:22.446108 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:22.454141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:22.454632 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:22.455287 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:22.456191 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:22.456651 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:22.457078 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:22.529834 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:22.538182 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-afafd853-93ab-4a15-8b18-3d4627c439f9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1284 microversion: 2.1 time: 0.097270 Sep 11 01:05:22.539317 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 524/1066] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:22 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1284 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:23.562231 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:23.562984 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:23.570330 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:23.570849 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:23.571291 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:23.571992 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:23.572457 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:23.572883 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:23.768033 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:23.776117 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-135648f9-865d-401d-9d40-ec73e09696d9 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1314 microversion: 2.1 time: 0.218304 Sep 11 01:05:23.776967 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 543/1067] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:23 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1314 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:23.806871 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [req-1b9ca2e7-9232-4347-abb3-cc5eb2e5239f req-af33841d-2b65-46c7-b487-d95b8bd1820b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1353237c-5669-4f7f-b7ec-bfa2c33417b3", "tag": "21c27e92-c859-4c34-8e76-5dc0618562bf"}]} {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:23.818000 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-1b9ca2e7-9232-4347-abb3-cc5eb2e5239f req-af33841d-2b65-46c7-b487-d95b8bd1820b service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:23.818583 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-1b9ca2e7-9232-4347-abb3-cc5eb2e5239f req-af33841d-2b65-46c7-b487-d95b8bd1820b service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:23.819109 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [req-1b9ca2e7-9232-4347-abb3-cc5eb2e5239f req-af33841d-2b65-46c7-b487-d95b8bd1820b service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:23.830368 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.compute.server_external_events [req-1b9ca2e7-9232-4347-abb3-cc5eb2e5239f req-af33841d-2b65-46c7-b487-d95b8bd1820b service nova] Creating event network-changed:21c27e92-c859-4c34-8e76-5dc0618562bf for instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 on hv-cinder-maste Sep 11 01:05:23.835254 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [req-1b9ca2e7-9232-4347-abb3-cc5eb2e5239f req-af33841d-2b65-46c7-b487-d95b8bd1820b service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032122 Sep 11 01:05:23.835836 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 525/1068] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:05:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:05:24.800444 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:24.801065 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:24.807409 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:24.807885 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:24.808310 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:24.808986 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:24.809432 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:24.809897 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:24.965613 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:24.974031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-11ef7c74-7acf-4dc8-a3d4-923308b6f29c tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1314 microversion: 2.1 time: 0.177410 Sep 11 01:05:24.974765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 544/1069] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:24 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1314 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:25.999594 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:26.000292 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:26.009246 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:26.010131 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:26.010967 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:26.012238 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:26.013062 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:26.013531 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:26.145450 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:26.153125 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-05885726-36d5-41bf-9005-b00f38d1450a tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1314 microversion: 2.1 time: 0.158938 Sep 11 01:05:26.154420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 526/1070] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:25 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:27.176709 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:27.177340 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:27.183958 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:27.184384 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:27.185699 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:27.186128 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:27.186562 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:27.187089 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:27.318527 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:27.326594 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-59c3a416-d89a-415e-8a4e-845ba7a7fb1d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1314 microversion: 2.1 time: 0.153770 Sep 11 01:05:27.327187 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 545/1071] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:27 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:28.349281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:28.349888 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:28.356539 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:28.357033 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:28.357490 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:28.358230 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:28.358679 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:28.359124 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:28.489420 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:28.496955 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-0b808394-c0d1-4b25-9092-10382daba80d tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1314 microversion: 2.1 time: 0.151509 Sep 11 01:05:28.498078 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 527/1072] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:28 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:29.272692 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-12dbf152-2ef8-4ca1-b0fc-76597503ea06 req-26127453-e647-4b84-bb6e-a9cf47d9e2a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1353237c-5669-4f7f-b7ec-bfa2c33417b3", "name": "network-vif-plugged", "status": "completed", "tag": "21c27e92-c859-4c34-8e76-5dc0618562bf"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:29.284137 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-12dbf152-2ef8-4ca1-b0fc-76597503ea06 req-26127453-e647-4b84-bb6e-a9cf47d9e2a1 service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:29.284513 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-12dbf152-2ef8-4ca1-b0fc-76597503ea06 req-26127453-e647-4b84-bb6e-a9cf47d9e2a1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:29.284942 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-12dbf152-2ef8-4ca1-b0fc-76597503ea06 req-26127453-e647-4b84-bb6e-a9cf47d9e2a1 service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:29.296629 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-12dbf152-2ef8-4ca1-b0fc-76597503ea06 req-26127453-e647-4b84-bb6e-a9cf47d9e2a1 service nova] Creating event network-vif-plugged:21c27e92-c859-4c34-8e76-5dc0618562bf for instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 on hv-cinder-maste Sep 11 01:05:29.301568 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-12dbf152-2ef8-4ca1-b0fc-76597503ea06 req-26127453-e647-4b84-bb6e-a9cf47d9e2a1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032598 Sep 11 01:05:29.302152 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 546/1073] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:05:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:05:29.520886 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:29.521820 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:29.529086 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:29.529911 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:29.530351 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:29.531071 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:29.531641 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:29.532066 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:29.666064 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:29.673688 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-4aefbe37-c55f-4dab-b5e5-4909192cca24 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1314 microversion: 2.1 time: 0.157567 Sep 11 01:05:29.674250 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 528/1074] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:29 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:30.697427 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:30.698042 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:30.704501 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:30.704982 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:30.705414 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:30.706094 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:30.706546 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:30.707070 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:30.834689 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:30.843058 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-a7e9722e-542e-48f8-bac3-6b2084d7407b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1498 microversion: 2.1 time: 0.150404 Sep 11 01:05:30.843624 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 547/1075] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:30 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1498 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:31.342897 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:31.343423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:31.345765 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:31.346423 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell0:SELECT=6 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:31.352391 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:31.352889 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:31.353335 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:31.354035 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:31.354496 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:31.354954 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:31.381544 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Going to try to terminate instance {{(pid=126979) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 11 01:05:31.457027 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-d84744be-a39b-404b-b6a6-fa8a7fd28120 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "DELETE /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 204 len: 0 microversion: 2.1 time: 0.118451 Sep 11 01:05:31.457684 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 529/1076] 10.222.0.27 () {58 vars in 1239 bytes} [Sun Sep 11 01:05:31 2022] DELETE /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 11 01:05:31.466498 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:31.467181 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:31.469114 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG dbcounter [-] [126981] Writing DB stats nova_cell0:SELECT=2 {{(pid=126981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:31.474536 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:31.475053 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:31.475495 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:31.476259 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:31.476828 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:31.477284 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:31.609456 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.policy [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126981) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:31.617184 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-aa70b1aa-61da-413d-ac40-f755ae9d536b tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1504 microversion: 2.1 time: 0.155056 Sep 11 01:05:31.617743 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 548/1077] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:31 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1504 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:32.641486 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:32.642141 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:32.648546 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:32.649026 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:32.649459 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:32.650148 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:32.650603 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:32.651053 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:32.655100 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG dbcounter [-] [126979] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=126979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:05:32.784215 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:32.791869 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-3785f225-2f8b-47a7-ad19-2c09720b9d4f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1504 microversion: 2.1 time: 0.154922 Sep 11 01:05:32.792412 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 530/1078] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:32 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1504 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:33.356959 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [req-8f5d3ee2-bf5b-4a6f-ae68-410033797e82 req-4eefc444-ef32-415b-b7e6-2ebc3410b22e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1353237c-5669-4f7f-b7ec-bfa2c33417b3", "name": "network-vif-deleted", "tag": "21c27e92-c859-4c34-8e76-5dc0618562bf"}]} {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 11 01:05:33.367798 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-8f5d3ee2-bf5b-4a6f-ae68-410033797e82 req-4eefc444-ef32-415b-b7e6-2ebc3410b22e service nova] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:33.368277 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-8f5d3ee2-bf5b-4a6f-ae68-410033797e82 req-4eefc444-ef32-415b-b7e6-2ebc3410b22e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:33.368703 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [req-8f5d3ee2-bf5b-4a6f-ae68-410033797e82 req-4eefc444-ef32-415b-b7e6-2ebc3410b22e service nova] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:33.388490 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.compute.server_external_events [req-8f5d3ee2-bf5b-4a6f-ae68-410033797e82 req-4eefc444-ef32-415b-b7e6-2ebc3410b22e service nova] Creating event network-vif-deleted:21c27e92-c859-4c34-8e76-5dc0618562bf for instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 on hv-cinder-maste Sep 11 01:05:33.393422 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [req-8f5d3ee2-bf5b-4a6f-ae68-410033797e82 req-4eefc444-ef32-415b-b7e6-2ebc3410b22e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040480 Sep 11 01:05:33.394011 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 549/1079] 10.222.0.27 () {64 vars in 1317 bytes} [Sun Sep 11 01:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 11 01:05:33.815326 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.api.openstack.wsgi [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126979) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:33.815947 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.compute.api [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126979) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:33.822324 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:33.822843 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:33.823281 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:33.824006 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:33.824453 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:33.824865 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:33.857604 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:33.858162 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:33.858599 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG oslo_concurrency.lockutils [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126979) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:33.859075 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.objects.instance [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lazy-loading 'fault' on Instance uuid 1353237c-5669-4f7f-b7ec-bfa2c33417b3 {{(pid=126979) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 11 01:05:33.903839 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: DEBUG nova.policy [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f63f7c17ba29451b8ba9599636c17e67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74dc2b85295647929cd675c10ea882c7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=126979) authorize /opt/stack/nova/nova/policy.py:199}} Sep 11 01:05:33.911957 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: INFO nova.api.openstack.requestlog [None req-f5ab19e3-db4e-4ad9-a808-80fc80e0ce9f tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 200 len: 1304 microversion: 2.1 time: 0.100399 Sep 11 01:05:33.912709 dv-cinder-master-3-iscsi devstack@n-api.service[126979]: [pid: 126979|app: 0|req: 531/1080] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:33 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 1304 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 11 01:05:34.936504 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Calling method '>' {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 11 01:05:34.937192 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.compute.api [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] [instance: 1353237c-5669-4f7f-b7ec-bfa2c33417b3] Fetching instance by UUID {{(pid=126981) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 11 01:05:34.943290 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:34.943765 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:34.944193 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:34.944873 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Acquiring lock "dae7eaab-b968-41e0-b510-529370888821" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 11 01:05:34.945318 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 11 01:05:34.945734 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG oslo_concurrency.lockutils [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Lock "dae7eaab-b968-41e0-b510-529370888821" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=126981) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 11 01:05:34.959387 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.wsgi [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] HTTP exception thrown: Instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 could not be found. Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Request method failure captured: Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: request: GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 HTTP/1.1 Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept: application/json Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Accept-Encoding: identity Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Connection: close Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Length: 0 Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Content-Type: application/json Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Host: 10.222.0.27 Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: Openstack-System-Scope: None Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: User-Agent: python-urllib3/1.26.12 Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Auth-Token: gAAAAABjHTRANQYY5al30RkAIWxXgCCXgmc46awUuThjf1OBsfMox6Dv_YZzbDMbWCw5n7n_7JNLHMM1uulGXR2L935P5LiIYkB9S7s0weGSK88PbEh15KOoTVWlaBp8t2HGwm3BA4O4SngPn5K-Ypfh6ZjbtfNi7ChT5IDb6F3d1I0dghtM3zc Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Id: None Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Domain-Name: None Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Identity-Status: Confirmed Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Is-Admin-Project: True Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Id: default Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Domain-Name: Default Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Id: 74dc2b85295647929cd675c10ea882c7 Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Project-Name: tempest-VolumesActionsTest-1078823862 Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Role: member,reader Sep 11 01:05:34.961031 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Roles: member,reader Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/74dc2b85295647929cd675c10ea882c7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/74dc2b85295647929cd675c10ea882c7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/74dc2b85295647929cd675c10ea882c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.27:8080", "publicURL": "http://10.222.0.27:8080/v1/AUTH_74dc2b85295647929cd675c10ea882c7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}] Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant: tempest-VolumesActionsTest-1078823862 Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Id: 74dc2b85295647929cd675c10ea882c7 Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-Tenant-Name: tempest-VolumesActionsTest-1078823862 Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User: tempest-VolumesActionsTest-1078823862-project Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Id: default Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Domain-Name: Default Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Id: f63f7c17ba29451b8ba9599636c17e67 Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: X-User-Name: tempest-VolumesActionsTest-1078823862-project Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: method: > Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: exception: Instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 could not be found. Sep 11 01:05:34.962871 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: action_args: {'id': '1353237c-5669-4f7f-b7ec-bfa2c33417b3'} Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: {{(pid=126981) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 could not be found. Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 11 01:05:34.964173 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 could not be found. Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 could not be found. Sep 11 01:05:34.966572 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: ERROR nova.api.openstack.wsgi Sep 11 01:05:34.968553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: DEBUG nova.api.openstack.wsgi [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] Returning 404 to user: Instance 1353237c-5669-4f7f-b7ec-bfa2c33417b3 could not be found. {{(pid=126981) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 11 01:05:34.968553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: INFO nova.api.openstack.requestlog [None req-2aec3caa-b4bb-4b77-a500-6cf84a63b884 tempest-VolumesActionsTest-1078823862 tempest-VolumesActionsTest-1078823862-project] 10.222.0.27 "GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3" status: 404 len: 111 microversion: 2.1 time: 0.030907 Sep 11 01:05:34.968553 dv-cinder-master-3-iscsi devstack@n-api.service[126981]: [pid: 126981|app: 0|req: 550/1081] 10.222.0.27 () {58 vars in 1236 bytes} [Sun Sep 11 01:05:34 2022] GET /compute/v2.1/servers/1353237c-5669-4f7f-b7ec-bfa2c33417b3 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)